Posts

Showing posts from September, 2022

How to bulk import NUL (0x0, ascii 0) character from csv to BigQuery

Image
If you ever exported a csv file from MsSql via bcp you probably got a csv file filled with NUL chars When using bq load you get this error  BigQuery error in load operation: Error processing job '[project]:[job_id]': Error while reading data, error message: Error detected while parsing row starting at position: 0. Error: Bad character (ASCII 0) encountered . File: gs://[bucket]/[path]/[file].csv Failure details: - Error while reading data, error message: CSV processing encountered One solution is to let bq load these characters using this flag --preserve_ascii_control_characters=true Note that the unicode general "unknown character"  �  symbol is inserted instead of null and you may want to clean this post bulk. But at least bq load does not crash.