Skip to main content

DBT Learning -2 - snowflake with data

 

You need to have a Snowflake Data platform for it . If not please create one as below:

You can start by signing up for a free trial on Snowflake:

Sign up for a free trial by following link https://signup.snowflake.com/ and completing the sign-up form.

Select the Enterprise edition, choose a cloud provider and region, and agree to the terms of service.

Click GET STARTED.

After submitting the sign-up form, you should receive an email asking you to activate your account. Click the link in the email and a new tab will open up where you’ll create your username and password. Complete the form and click Get started.

Congrats! Your workspace is ready for some data.  Now to load sample data use below queries.

#creating Warehouse, Database and schemas

create warehouse transforming;

create database raw;

create database analytics;

create schema raw.jaffle_shop;

create schema raw.stripe;

#creating customer table and loading Data

create table raw.jaffle_shop.customers 

( id integer,

  first_name varchar,

  last_name varchar

);

copy into raw.jaffle_shop.customers (id, first_name, last_name)

from 's3://dbt-tutorial-public/jaffle_shop_customers.csv'

file_format = (

  type = 'CSV'

  field_delimiter = ','

  skip_header = 1

  );

#creating orders table and loading data

create table raw.jaffle_shop.orders

( id integer,

  user_id integer,

  order_date date,

  status varchar,

  _etl_loaded_at timestamp default current_timestamp

);

copy into raw.jaffle_shop.orders (id, user_id, order_date, status)

from 's3://dbt-tutorial-public/jaffle_shop_orders.csv'

file_format = (

  type = 'CSV'

  field_delimiter = ','

  skip_header = 1

  );

#creaing payments table and loading data

create table raw.stripe.payment 

( id integer,

  orderid integer,

  paymentmethod varchar,

  status varchar,

  amount integer,

  created date,

  _batched_at timestamp default current_timestamp

);

copy into raw.stripe.payment (id, orderid, paymentmethod, status, amount, created)

from 's3://dbt-tutorial-public/stripe_payments.csv'

file_format = (

  type = 'CSV'

  field_delimiter = ','

  skip_header = 1

  );


#validating the loaded data

select * from raw.jaffle_shop.customers;
select * from raw.jaffle_shop.orders;
select * from raw.stripe.payment;

Comments

Popular posts from this blog

Gateway port to use 80 to 443 on Tableau Server

Change Gateway port from 80 to 443 or 8080       To change the Gateway port from 80 to 443 then use below commands First , list your nodes with the command. tsm topology list-node  Then execute the command to set port  tsm topology set-ports --node-name node1 --port-name gateway:primary --port-value 443 The above command will ask for a restart.  Check ports using  tsm topology list-ports  If some error occurs in applying changes: try disabling (enable later) external ssl first- tsm security external-ssl disable You might need to run init command again with new port- tsm reset tabcmd initialuser --server localhost:443 --username 'ADMINUSER' --password 'adminpwd'

Creating Groups using Tableau API and Python Script

 Creating Tableau Groups using Python and API Prerequisites: Python on the machine tableauserverclient module API enabled on Tableau server. Permissions to update for the user Groups.csv file . list all groups in a csv file Changes to be made in the script for below lines in the script groups = open('<Filepath>/groups.csv') pat='getyourpersonaltoken' server = TSC.Server('http://myserver.tableau.com',use_server_version=True) tokenName = 'mytokenname' Script: import tableauserverclient as TSC import csv groups = open('<Filepath>/groups.csv') grp = csv.reader(groups) sourcegroups=[] createdgrps=[] pat='getyourpersonaltoken' server = TSC.Server('http://myserver.tableau.com',use_server_version=True) tokenName = 'mytokenname' ta = TSC.PersonalAccessTokenAuth(token_name=tokenName, personal_access_token=pat) with server.auth.sign_in_with_personal_access_token(ta):    groupslist, pagination_item  = server.groups.get() ...

Tableau dirty Backup

Steps for performing a dirty restore 1. Install Tableau Server 2019.1.0 on the Dev machine 2. Stop Tableau Server 3. Copy the following folders from the current production: D:\Tableau Server\data\tabsvc\pgsql D:\Tableau Server\data\tabsvc\dataengine 4. Update the pg_hba.conf D:\Tableau Server\data\tabsvc\config\pgsql_0.<version>/pg_hba.conf Change "md5" to "trust" for the user "tblwgadmin" like this: host    all         tblwgadmin         <address>/32          md5 to : host    all         tblwgadmin        <address>/32         trust 5. Regenerate the internal token: tsm security regenerate-internal-tokens [options] [global options] tsm pending-changes apply 6. Re-index tsm maintenance reindex-search 7. Make a proper backup: tsm maintenance backup -f myBackup At this point, we w...