Posts

Showing posts from 2022

How to bulk import NUL (0x0, ascii 0) character from csv to BigQuery

Image
If you ever exported a csv file from MsSql via bcp you probably got a csv file filled with NUL chars When using bq load you get this error  BigQuery error in load operation: Error processing job '[project]:[job_id]': Error while reading data, error message: Error detected while parsing row starting at position: 0. Error: Bad character (ASCII 0) encountered . File: gs://[bucket]/[path]/[file].csv Failure details: - Error while reading data, error message: CSV processing encountered One solution is to let bq load these characters using this flag --preserve_ascii_control_characters=true Note that the unicode general "unknown character"  �  symbol is inserted instead of null and you may want to clean this post bulk. But at least bq load does not crash.

Connect to Azure SQL using integrated (azure) security in powershell

  $ErrorActionPreference = "stop" $server = "your-server.database.windows.net" $database = "your-database" $query = "select 1 from dbo.yourtable" Connect-AzAccount $token = ( Get-AzAccessToken -ResourceUrl "https://database.windows.net" ).Token Invoke-Sqlcmd -server $server -Database $database -Query $query -as dataset -AccessToken $token write-output "Done!"

Connect to Azure SQL database using System Assigned Identity (integrated security / no passwords) from RunBook

This is how to grant user from system assigned identity (e.g. automation account) to allow to connect and query azure sql database without any use of keys, secrets, vaults or hard coded passwords. The app does not have to be contributor in azure sql instance for this to work Enable System Assigned Identity Navigate to Automation Account / Identity Set System Assigned: On Copy Object (Principal) Id guid Navigate to Azure Active Directory Search for the guid in "Search your tenant" box Find the "enterprise application" popping up Copy the Name this is what you use below [App_Name] Grant access to Azure SQL Database CREATE USER [App_Name] FROM EXTERNAL PROVIDER WITH DEFAULT_SCHEMA = dbo ALTER ROLE db_datareader ADD MEMBER [App_Name] ALTER ROLE db_datawriter ADD MEMBER [App_Name] How to use from runbook (powershell) $server  =  "your-sql-server-name.database.windows.net" $database  =  "your-database-name" $query  =  "sel...

SSAS Tabular .configsettings sample file

It's very tricky to find the schema of the crucial .configsettings file that is used when automating deployments of ssas projects.  It has come to my understanding that msbuild does not produce this file for Tabular Model projects. Though in some cases Microsoft forget to mention this in their guides.  I know clarification is pending on this page https://docs.microsoft.com/en-us/analysis-services/deployment/deployment-script-files-solution-deployment-config-settings?view=asallproducts-allversions Meanwhile. Here is a sample of the file in case you need it <ConfigurationSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2" xmlns:ddl100_100="http://schemas.microsoft.com/analysisservices/2008/engine/100/100" xmlns:ddl200="http:...