Skip to main content
    • Knowledge Base/
    Cannot Append Data To Parquet Format

    Resolving "Cannot Append Data in Parquet Format" Error in ClickHouse

    Are you getting the error "Cannot append data in format Parquet to file" error in ClickHouse? Let's take a look at how to resolve it.

    Resolving "Cannot Append Data in Parquet Format" Error in ClickHouse​

    Are you getting the error "Cannot append data in format Parquet to file" error in ClickHouse?

    Typically the error is reported as:

    DB::Exception: Cannot append data in format Parquet to file, because this format doesn't support appends. (CANNOT_APPEND_TO_FILE)

    Suppose you create a table that uses the File table engine with the Parquet format.

    You can write to the table once:

    This creates a file named data.Parquet in the data/default/parquet_test folder. If you try to insert another batch:

    ...you get the following error:

    You can not append to Parquet files in ClickHouse. But you can tell ClickHouse to create a new file for every INSERT by enabling the engine_file_allow_create_multiple_files setting. If enabled, on each insert a new file will be created with a name following this pattern:

    data.Parquet -> data.1.Parquet -> data.2.Parquet, etc.:

    Let's give it a try. We will put our two commands into a single file named parquet.sql:

    Run it using clickhouse-client:

    And now you will see two files in data/default/parquet_test (and a new file for each subsequent insert).

    Note

    The engine_file_allow_create_multiple_files setting applies to other data formats that are not appendable, like JSON and ORC.

    · 2 min read