1. marco kiesewettermarco kiesewetter
SIMPLE ETL
SOLUTION
Extracting,Transforming, Loading
Data From Any System To SQL
Server Right FromYour Desktop.

2. marco kiesewettermarco kiesewetter
OVERVIEW
 Why use manual ETL from your desktop?
 A common issue to deal with
 Example: Salesforce to SQL - a simple way to load fresh
data
 Recap:What is ETL?
 Extract data from outside source, i.e. Salesforce.com
 Transform data to fit operational needs
 Load data into a data storage system such as a SQL database, data mart or
data warehouse

3. marco kiesewettermarco kiesewetter
WHY USE MANUAL ETL
 Business Analysts, Financial Analysts and BI Developers run
into two common situations where a manual desktop
upload is needed:
 The data does not yet exist in SQL
 Often it is helpful to get a sample dataset into SQL to test and justify an
new data source that is needed.
 Development using this new data source can begin immediately, even before
the automated ETL job is set up by your IT department
 The data needs to be refreshed off-schedule
 Many ETL jobs run once or twice a day. If an extra refresh is needed at
times, a manual upload of ‘fresh’ data can be the quickest way

4. marco kiesewettermarco kiesewetter
A COMMON ISSUE TO DEAL WITH
One of the most common issues is the formatting
of the raw data
 Field delimiters & row delimiters may not be standard
 The use of quotations and commas in text fields can cause delimiter
recognition to fail
 SQL Server does only support CSV upload if the data is in a specific
format
 Salesforce row delimiters are not recognized
 Quotes only work if all fields in a column are enclosed in quotes

5. marco kiesewettermarco kiesewetter
SALESFORCE TO SQL - A SIMPLE WAY
TO LOAD FRESH DATA
Example

6. marco kiesewettermarco kiesewetter
AUTOMATION & SIMPLICITY
Requirement:
 Automation – very little manual effort
 Simplicity – anyone can run the update

7. marco kiesewettermarco kiesewetter
STEP 1: THE EXTRACT FILE
Download the Salesforce report results you want
to upload
 Download as CSV
 Save in an accessible network path
 The SQL server has to be able to access it.
 Use a filename that does not change
 Avoid dates etc. in the file name.
 Scripts will access this filename in this folder. For an update simply
overwrite this file.

8. marco kiesewettermarco kiesewetter
STEP 2: STANDARDIZE THE FILE
 Now we are usingWindows PowerShell to prepare the csv for upload
 Often we will have commas in comments or other text fields.We should change the
field delimiter to a different symbol.The pipe “|” is a good option.
 First we will need change all existing pipes to something else in order to make the
pipe symbol unique as field delimiter.
 In the example below I simply remove the pipes by replacing them with an empty
string, they can be replaced with anything else, though.
# Define the file but note that I do not add “.csv”
$csvfile = 'serverfoldersalesforceExtract'
# Now we replace all pipes
get-content ($csvfile + ".csv") | % {$_ -replace "|", ""} |
out-file ($csvfile +" (no Pipes).csv") -force -encoding ascii

9. marco kiesewettermarco kiesewetter
STEP 2: STANDARDIZE THE FILE
 Next we will change the delimiter and standardize the CSV file:
 The file salesforceExtract (standardized).csv has now all fields in quotes.
Since we do not use commas as delimiters and all pipes were removed
from all text fields, we can safely remove all quotes from the file:
# Make standard CSV but use Pipes as delimiter
Import-csv -path ($csvfile + " (no Pipes).csv") -Delimiter ','
| Export-CSV -path ($csvfile + " (standardized).csv") -
Delimiter '|'
# Remove all Quotes (“)
get-content ($csvfile + " (standardized).csv") | % { $_ -
replace '"',""} | Set-Content ($csvfile + " (upload).csv")

10. marco kiesewettermarco kiesewetter
STEP 3: UPLOAD TO SQL
 Finally, we upload this prepared CSV to SQL server using
Microsoft SQL Server Management Studio
 For this we use the BULK INSERT command
 In most cases we may upload a complete new data set.The easiest way to
handle this is by deleting the old table and re-creating it.
 Another advantage for doing this is the ease with which new fields can be
added or the variable types of fields can be changed.The new creation of
the table allows for any such adjustments.
 Note that the BULK INSERT functionality is a server-side permission
setting and may need to be activated for your login.

11. marco kiesewettermarco kiesewetter
STEP 3: UPLOAD TO SQL
 Here is an example SQL
script you can adjust to
fit your needs:
use YourDB
drop table [dbo].[YourTable]
go
create table [dbo].[YourTable](
Field1 nVARCHAR(255) null,
Field2 datetime null,
Field3 float null,
go
bulk insert YourTable
from 'serverfoldersalesfoceExtract
(upload).csv'
With (
fieldterminator = '|',
rowterminator = 'n',
firstrow = 2
)
go

12. marco kiesewettermarco kiesewetter
STEP 4: PUTTING IT ALLTOGETHER
 Put all the PowerShell commands into a text file with the extension .ps1
 Put the SQL script into a text file with the extension .sql
 Now create a batch script (example below, file extension .bat) that runs
all of the above commands and place everything in the same folder in
which you save your extracted CSV from Salesforce
@echo off
cls
echo Standardizing the CSV file...
powershell.exe -noprofile -ExecutionPolicy ByPass -File “My PowerShell
Script.ps1"
echo.
echo Is SQL Server Management Studio running and logged in to server ?
pause
echo.
echo Loading the SQL Query
“My SQL Script.sql"

13. marco kiesewettermarco kiesewetter
SIMPLE ETL
Your automated ETL solution is ready.
All you have to do now is saving the report results
under the same file name, run the batch script and
hit “Execute” in SQL Management Studio once it
loaded.

14. marco kiesewettermarco kiesewetter
THANKYOU
Questions?
Reach out:
https://www.linkedin.com/in/marcokiesewetter