cudf::io::chunked_parquet_writer_options Class Reference

Settings for write_parquet_chunked(). More...

#include <parquet.hpp>

Public Member Functions
	chunked_parquet_writer_options ()=default
	Default constructor. More...
sink_info const &	get_sink () const
	Returns sink info. More...
compression_type	get_compression () const
	Returns compression format used. More...
statistics_freq	get_stats_level () const
	Returns level of statistics requested in output file. More...
auto const &	get_metadata () const
	Returns metadata information. More...
std::vector< std::map< std::string, std::string > > const &	get_key_value_metadata () const
	Returns Key-Value footer metadata information. More...
bool	is_enabled_int96_timestamps () const
	Returns true if timestamps will be written as INT96. More...
auto	is_enabled_utc_timestamps () const
	Returns true if timestamps will be written as UTC. More...
auto	get_row_group_size_bytes () const
	Returns maximum row group size, in bytes. More...
auto	get_row_group_size_rows () const
	Returns maximum row group size, in rows. More...
auto	get_max_page_size_bytes () const
	Returns maximum uncompressed page size, in bytes. More...
auto	get_max_page_size_rows () const
	Returns maximum page size, in rows. More...
auto	get_column_index_truncate_length () const
	Returns maximum length of min or max values in column index, in bytes. More...
dictionary_policy	get_dictionary_policy () const
	Returns policy for dictionary use. More...
auto	get_max_dictionary_size () const
	Returns maximum dictionary size, in bytes. More...
auto	get_max_page_fragment_size () const
	Returns maximum page fragment size, in rows. More...
std::shared_ptr< writer_compression_statistics >	get_compression_statistics () const
	Returns a shared pointer to the user-provided compression statistics. More...
auto	is_enabled_write_v2_headers () const
	Returns true if V2 page headers should be written. More...
void	set_metadata (table_input_metadata metadata)
	Sets metadata. More...
void	set_key_value_metadata (std::vector< std::map< std::string, std::string >> metadata)
	Sets Key-Value footer metadata. More...
void	set_stats_level (statistics_freq sf)
	Sets the level of statistics in parquet_writer_options. More...
void	set_compression (compression_type compression)
	Sets compression type. More...
void	enable_int96_timestamps (bool req)
	Sets timestamp writing preferences. More...
void	enable_utc_timestamps (bool val)
	Sets preference for writing timestamps as UTC. Write timestamps as UTC if set to true. More...
void	set_row_group_size_bytes (size_t size_bytes)
	Sets the maximum row group size, in bytes. More...
void	set_row_group_size_rows (size_type size_rows)
	Sets the maximum row group size, in rows. More...
void	set_max_page_size_bytes (size_t size_bytes)
	Sets the maximum uncompressed page size, in bytes. More...
void	set_max_page_size_rows (size_type size_rows)
	Sets the maximum page size, in rows. More...
void	set_column_index_truncate_length (int32_t size_bytes)
	Sets the maximum length of min or max values in column index, in bytes. More...
void	set_dictionary_policy (dictionary_policy policy)
	Sets the policy for dictionary use. More...
void	set_max_dictionary_size (size_t size_bytes)
	Sets the maximum dictionary size, in bytes. More...
void	set_max_page_fragment_size (size_type size_rows)
	Sets the maximum page fragment size, in rows. More...
void	set_compression_statistics (std::shared_ptr< writer_compression_statistics > comp_stats)
	Sets the pointer to the output compression statistics. More...
void	enable_write_v2_headers (bool val)
	Sets preference for V2 page headers. Write V2 page headers if set to true. More...

Static Public Member Functions
static chunked_parquet_writer_options_builder	builder (sink_info const &sink)
	creates builder to build chunked_parquet_writer_options. More...

Detailed Description

Settings for write_parquet_chunked().

Definition at line 1189 of file parquet.hpp.

Constructor & Destructor Documentation

chunked_parquet_writer_options()

cudf::io::chunked_parquet_writer_options::chunked_parquet_writer_options ( )

default

Default constructor.

This has been added since Cython requires a default constructor to create objects on stack.

Member Function Documentation

builder()

static chunked_parquet_writer_options_builder cudf::io::chunked_parquet_writer_options::builder ( sink_info const &  sink )

static

creates builder to build chunked_parquet_writer_options.

Parameters

sink	sink to use for writer output

Returns

Builder to build chunked_parquet_writer_options

enable_int96_timestamps()

void cudf::io::chunked_parquet_writer_options::enable_int96_timestamps ( bool  req )

inline

Sets timestamp writing preferences.

INT96 timestamps will be written if true and TIMESTAMP_MICROS will be written if false.

Parameters

req	Boolean value to enable/disable writing of INT96 timestamps

Definition at line 1414 of file parquet.hpp.

enable_utc_timestamps()

void cudf::io::chunked_parquet_writer_options::enable_utc_timestamps ( bool  val )

inline

Sets preference for writing timestamps as UTC. Write timestamps as UTC if set to true.

Parameters

val	Boolean value to enable/disable writing of timestamps as UTC.

Definition at line 1421 of file parquet.hpp.

enable_write_v2_headers()

void cudf::io::chunked_parquet_writer_options::enable_write_v2_headers ( bool  val )

inline

Sets preference for V2 page headers. Write V2 page headers if set to true.

Parameters

val	Boolean value to enable/disable writing of V2 page headers.

Definition at line 1494 of file parquet.hpp.

get_column_index_truncate_length()

auto cudf::io::chunked_parquet_writer_options::get_column_index_truncate_length ( ) const

inline

Returns maximum length of min or max values in column index, in bytes.

Returns

length min/max will be truncated to

Definition at line 1339 of file parquet.hpp.

get_compression()

compression_type cudf::io::chunked_parquet_writer_options::get_compression ( ) const

inline

Returns compression format used.

Returns

Compression format

Definition at line 1255 of file parquet.hpp.

get_compression_statistics()

std::shared_ptr<writer_compression_statistics> cudf::io::chunked_parquet_writer_options::get_compression_statistics ( ) const

inline

Returns a shared pointer to the user-provided compression statistics.

Returns

Compression statistics

Definition at line 1367 of file parquet.hpp.

get_dictionary_policy()

dictionary_policy cudf::io::chunked_parquet_writer_options::get_dictionary_policy ( ) const

inline

Returns policy for dictionary use.

Returns

policy for dictionary use

Definition at line 1346 of file parquet.hpp.

get_key_value_metadata()

std::vector<std::map<std::string, std::string> > const& cudf::io::chunked_parquet_writer_options::get_key_value_metadata ( ) const

inline

Returns Key-Value footer metadata information.

Returns

Key-Value footer metadata information

Definition at line 1276 of file parquet.hpp.

get_max_dictionary_size()

auto cudf::io::chunked_parquet_writer_options::get_max_dictionary_size ( ) const

inline

Returns maximum dictionary size, in bytes.

Returns

Maximum dictionary size, in bytes.

Definition at line 1353 of file parquet.hpp.

get_max_page_fragment_size()

auto cudf::io::chunked_parquet_writer_options::get_max_page_fragment_size ( ) const

inline

Returns maximum page fragment size, in rows.

Returns

Maximum page fragment size, in rows.

Definition at line 1360 of file parquet.hpp.

get_max_page_size_bytes()

auto cudf::io::chunked_parquet_writer_options::get_max_page_size_bytes ( ) const

inline

Returns maximum uncompressed page size, in bytes.

If set larger than the row group size, then this will return the row group size.

Returns

Maximum uncompressed page size, in bytes

Definition at line 1317 of file parquet.hpp.

get_max_page_size_rows()

auto cudf::io::chunked_parquet_writer_options::get_max_page_size_rows ( ) const

inline

Returns maximum page size, in rows.

If set larger than the row group size, then this will return the row group size.

Returns

Maximum page size, in rows

Definition at line 1329 of file parquet.hpp.

get_metadata()

auto const& cudf::io::chunked_parquet_writer_options::get_metadata ( ) const

inline

Returns metadata information.

Returns

Metadata information

Definition at line 1269 of file parquet.hpp.

get_row_group_size_bytes()

auto cudf::io::chunked_parquet_writer_options::get_row_group_size_bytes ( ) const

inline

Returns maximum row group size, in bytes.

Returns

Maximum row group size, in bytes

Definition at line 1300 of file parquet.hpp.

get_row_group_size_rows()

auto cudf::io::chunked_parquet_writer_options::get_row_group_size_rows ( ) const

inline

Returns maximum row group size, in rows.

Returns

Maximum row group size, in rows

Definition at line 1307 of file parquet.hpp.

get_sink()

sink_info const& cudf::io::chunked_parquet_writer_options::get_sink ( ) const

inline

Returns sink info.

Returns

Sink info

Definition at line 1248 of file parquet.hpp.

get_stats_level()

statistics_freq cudf::io::chunked_parquet_writer_options::get_stats_level ( ) const

inline

Returns level of statistics requested in output file.

Returns

Level of statistics requested in output file

Definition at line 1262 of file parquet.hpp.

is_enabled_int96_timestamps()

bool cudf::io::chunked_parquet_writer_options::is_enabled_int96_timestamps ( ) const

inline

Returns true if timestamps will be written as INT96.

Returns

true if timestamps will be written as INT96

Definition at line 1286 of file parquet.hpp.

is_enabled_utc_timestamps()

auto cudf::io::chunked_parquet_writer_options::is_enabled_utc_timestamps ( ) const

inline

Returns true if timestamps will be written as UTC.

Returns

true if timestamps will be written as UTC

Definition at line 1293 of file parquet.hpp.

is_enabled_write_v2_headers()

auto cudf::io::chunked_parquet_writer_options::is_enabled_write_v2_headers ( ) const

inline

Returns true if V2 page headers should be written.

Returns

true if V2 page headers should be written.

Definition at line 1377 of file parquet.hpp.

set_column_index_truncate_length()

void cudf::io::chunked_parquet_writer_options::set_column_index_truncate_length

(

int32_t

size_bytes

)

Sets the maximum length of min or max values in column index, in bytes.

Parameters

size_bytes

length min/max will be truncated to

set_compression()

void cudf::io::chunked_parquet_writer_options::set_compression ( compression_type  compression )

inline

Sets compression type.

Parameters

compression

The compression type to use

Definition at line 1405 of file parquet.hpp.

set_compression_statistics()

void cudf::io::chunked_parquet_writer_options::set_compression_statistics ( std::shared_ptr< writer_compression_statistics >  comp_stats )

inline

Sets the pointer to the output compression statistics.

Parameters

comp_stats

Pointer to compression statistics to be updated after writing

Definition at line 1484 of file parquet.hpp.

set_dictionary_policy()

void cudf::io::chunked_parquet_writer_options::set_dictionary_policy

(

dictionary_policy

policy

)

Sets the policy for dictionary use.

Parameters

policy

Policy for dictionary use

set_key_value_metadata()

void cudf::io::chunked_parquet_writer_options::set_key_value_metadata

(

std::vector< std::map< std::string, std::string >>

metadata

)

Sets Key-Value footer metadata.

Parameters

metadata

Key-Value footer metadata

set_max_dictionary_size()

void cudf::io::chunked_parquet_writer_options::set_max_dictionary_size

(

size_t

size_bytes

)

Sets the maximum dictionary size, in bytes.

Parameters

size_bytes

Maximum dictionary size, in bytes

set_max_page_fragment_size()

void cudf::io::chunked_parquet_writer_options::set_max_page_fragment_size

(

size_type

size_rows

)

Sets the maximum page fragment size, in rows.

Parameters

size_rows

Maximum page fragment size, in rows.

set_max_page_size_bytes()

void cudf::io::chunked_parquet_writer_options::set_max_page_size_bytes

(

size_t

size_bytes

)

Sets the maximum uncompressed page size, in bytes.

Parameters

size_bytes

Maximum uncompressed page size, in bytes to set

set_max_page_size_rows()

void cudf::io::chunked_parquet_writer_options::set_max_page_size_rows

(

size_type

size_rows

)

Sets the maximum page size, in rows.

Parameters

size_rows

The maximum page size, in rows to set

set_metadata()

void cudf::io::chunked_parquet_writer_options::set_metadata ( table_input_metadata  metadata )

inline

Sets metadata.

Parameters

metadata

Associated metadata

Definition at line 1384 of file parquet.hpp.

set_row_group_size_bytes()

void cudf::io::chunked_parquet_writer_options::set_row_group_size_bytes

(

size_t

size_bytes

)

Sets the maximum row group size, in bytes.

Parameters

size_bytes

Maximum row group size, in bytes to set

set_row_group_size_rows()

void cudf::io::chunked_parquet_writer_options::set_row_group_size_rows

(

size_type

size_rows

)

Sets the maximum row group size, in rows.

Parameters

size_rows

The maximum row group size, in rows to set

set_stats_level()

void cudf::io::chunked_parquet_writer_options::set_stats_level ( statistics_freq  sf )

inline

Sets the level of statistics in parquet_writer_options.

Parameters

sf	Level of statistics requested in the output file

Definition at line 1398 of file parquet.hpp.

---

The documentation for this class was generated from the following file:

• parquet.hpp