Parquet Import

Import Parquet files into DB9 tables using COPY ... WITH (FORMAT parquet), either from an HTTP URL or from a file stored in fs9 using the fs9:// scheme. No external tools or ETL pipelines are required.

Installation

CREATE EXTENSION IF NOT EXISTS parquet;

The unqualified read_parquet() is not registered as a table-valued function and returns:

ERROR: function read_parquet(unknown) does not exist   -- sqlstate 42883

Call it schema-qualified instead — extensions.read_parquet(...) works:

SELECT count(*) FROM extensions.read_parquet('https://example.com/data.parquet');

You can also use COPY FROM ... WITH (FORMAT parquet) to import Parquet data into an existing table — see below.

COPY FROM Parquet

Use COPY ... WITH (FORMAT parquet) to import Parquet data into an existing table, from either source:

-- Import from an HTTP URL
COPY my_table FROM 'https://example.com/export.parquet' WITH (FORMAT parquet);

-- Import from a Parquet file stored in fs9
COPY my_table FROM 'fs9:///data/export.parquet' WITH (FORMAT parquet);

Columns are matched by name (case-insensitive). Extra columns in the Parquet file are ignored, and target-table columns with no matching Parquet column are filled with NULL.

Many incompatible column pairs are caught at import (cannot cast type X to Y, or invalid input syntax for type X), but numeric source columns are not checked against non-numeric targets. Importing a numeric Parquet column (INT32, INT64, FLOAT, DOUBLE) into a uuid, date, time, timestamp, timestamptz, interval, inet or bytea column currently succeeds and stores the raw value, with no error — a uuid column ends up holding integers. Text source columns are checked against these same targets and rejected where the cast is genuinely invalid (TEXT into bytea is a valid cast and is accepted).

Treat that list as indicative rather than exhaustive: verify the Parquet schema matches your target table rather than relying on the import to reject a mismatch. Tracked in db9-server#4059.

Use the fs9:// scheme for fs9 files

Sources must be a URL. http://, https:// and fs9:// are accepted; a bare path is not:

ERROR: unsupported Parquet URL scheme

Write fs9:///data/export.parquet (three slashes — fs9:// plus the absolute fs9 path) rather than /data/export.parquet. The same rule applies to extensions.read_parquet().

Type Mapping

Parquet physical types are mapped to PostgreSQL types during COPY FROM:

Parquet Type	PostgreSQL Type
`BOOLEAN`	`BOOLEAN`
`INT32`	`INTEGER`
`INT64`	`BIGINT`
`FLOAT`	`REAL`
`DOUBLE`	`DOUBLE PRECISION`
`BYTE_ARRAY` (UTF8)	`TEXT`
`BYTE_ARRAY` (binary)	`BYTEA`
`INT96` (timestamp)	`TIMESTAMP`
`DATE` logical type	`DATE`
`TIME` logical type	`TIME`
`TIMESTAMP` logical type	`TIMESTAMP`
`DECIMAL` logical type	`NUMERIC`

Limits

Limit	Value
Max Parquet file size (`fs9://` sources)	100 MB
Supported Parquet versions	v1, v2
Max columns per file	Unlimited (practical limit: storage)
Remote URL connect timeout	10 seconds
Remote URL request timeout	60 seconds
Supported compression codecs	Snappy, Gzip, Zstd, uncompressed

See Limits and Quotas for the complete list.

Error Messages

Most errors fall into two groups. Those raised while fetching the source do not echo the URL or path, so the message alone does not identify the file — check the statement’s source argument. Those raised while importing rows do name the table, row group and row. The first row below is neither: it is a function-resolution error raised before either stage.

Error	Cause
`ERROR: function read_parquet(unknown) does not exist` (`42883`)	Unqualified `read_parquet()` is not on the default `search_path`. Call `extensions.read_parquet(...)`, or use `COPY FROM`. See the note below — the unqualified call does work in the session that ran `CREATE EXTENSION`.
`ERROR: Parquet file was not found` (`58P01`)	The remote URL returned a 404. Verify the URL is accessible.
`ERROR: could not resolve Parquet file host` (`58030`)	The host could not be resolved. Check the hostname.
`ERROR: could not connect to Parquet file host` (`58030`)	Connection failed or timed out.
`ERROR: Parquet HTTP response has no valid Content-Length` (`58030`)	The URL answered `200` but sent no usable `Content-Length` — commonly an HTML page rather than a file.
`ERROR: internal error` (`XX000`)	The body was fetched but is not a valid Parquet file, or is truncated.
`ERROR: Parquet URL is not allowed by network policy` (`42501`)	The URL points at a private, loopback or otherwise blocked address. Use a publicly reachable host, or `fs9://`.
`ERROR: Parquet file host returned HTTP status <code>` (`58030`)	The host answered with an unexpected status (for example `500`).
`ERROR: permission denied for Parquet file` (`42501`)	The remote URL returned 401 or 403.
`ERROR: COPY <table>, row group N, row M: cannot cast type X to Y`	A Parquet column’s type cannot be cast to the target column’s type at all (for example `INTEGER` into `BOOLEAN`).
`ERROR: COPY <table>, row group N, row M: invalid input syntax for type X: "..."`	The target type is parseable in principle but this value is not — for example `TEXT` into `INTEGER`, or `DOUBLE` into `INTEGER` (`"1.5"`). Note the reported row is the first offending row, not necessarily row 1.
`ERROR: unsupported Parquet URL scheme` (`22023`)	The source was a bare path, or used a scheme other than `http`, `https` or `fs9`. Use `fs9:///path` for fs9 files, or an `http(s)://` URL.
`ERROR: fs9 resource was not found` (`58P01`)	The `fs9://` path does not exist.
`ERROR: fs9 resource has the wrong object type` (`42809`)	The `fs9://` path exists but is not a file — most often a directory.

Next Steps

HTTP from SQL — Fetch Parquet files from external HTTP endpoints
Extensions Overview — All 9 built-in extensions
Limits and Quotas — All operational limits