Open a Parquet file

Opens a Parquet file for inexpensive metadata inspection and selective, batched reading. The returned handle is valid only in the current R session.

Usage

open_parquet(file, mmap = FALSE, verify_checksums = TRUE, threads = NULL)

Arguments

file

Path to a Parquet file, or an http://, https://, ftp://, ftps:// or file:// URL. A URL is downloaded to the session temporary directory in full before any of it is read; the copy is removed by close_parquet(), so a handle opened from a URL must be closed to reclaim the space. See qio-limitations.

mmap

Use memory-mapped input. On Windows a path the active code page cannot represent is read with buffered input instead, because only the mapped path needs a name that page can express. The result is the same.

verify_checksums

Verify Parquet page checksums when present.

threads

Number of reader threads. NULL (the default) and 0 both use two threads. Larger explicit values are honored in ordinary use and capped at two when R requests CRAN-compatible core limits. collect() decodes columns in parallel either way: a mapped file shares one reader, and a buffered one gives each worker its own. Pass threads = 1 to force serial reads.

Value

A qio_parquet_file object. Close it with close_parquet().

See also

collect() and walk_batches() to read from the handle, close_parquet() to release it, schema() and metadata() to inspect it without reading, and read_parquet() for a whole file in one call.

Examples

path <- tempfile(fileext = ".parquet")
write_parquet(mtcars, path)
pf <- open_parquet(path)
dim(pf)
#> [1] 32 11
close_parquet(pf)