SQL Server
(1)
Money
(1)
Database
(1)
Welch
(1)
Buckets
(1)
Jwelch
(1)

SSIS Package to Export SQL Server to Multiple Flat Files (based on size of row count)

Asked By JPo
18-Nov-09 06:49 PM
I have the need to create an export job that will query a large amount
of data from a SQL Server database and produce a flat file.  Because
of the amount of data, I would like to create a maximum file size or
record count for each flat file - for example a query with 5M records
would produce 5 flat files, each with 1M records.

Can this be done within an SSIS package?  I only have experience with
the simple import/export wizard.

Thank you in advance for any help!

I used to tell my COnsulting Clients:"Anything can be done with time and money.

Todd C replied to JPo
18-Nov-09 03:41 PM
I used to tell my COnsulting Clients:

If you are new to SSIS, I would not recommend attempting this in its
entirety. Start with small steps, maybe generting a single flat file from the
data.

Once you have that, experiment with how you would break up the source data
into equal buckets using WHERE clauses. Same those 'boundry' values in a SQL
table. Then set up a For Each that loops over the values and populates
variables. Inside the loop, have a parameterized SQL statement as the Source
in your data flow.

You might find some useful stuff at SQLIS.com or my blog:
http://toddchitt.wordpress.com/

Good Luck
=====
Todd C

Yes, this can be done, and John Welch has two great blog posts on

Phil Brammer replied to JPo
24-Nov-09 12:42 PM
Yes, this can be done, and John Welch has two great blog posts on this
topic:

Part 1 (requires scripting) -
http://agilebi.com/cs/blogs/jwelch/archive/2007/06/03/multi-file-output-destination-script-component.aspx
Part 2 (does not require scripting) -
http://agilebi.com/cs/blogs/jwelch/archive/2007/06/03/multi-file-outputs-part-2.aspx

Phil
ssistalk.com
Post Question To EggHeadCafe
SQL Server SQL SERVER DATABASE CREATION ERROR'S I have a new server we are installing Sql STANDARD 2005 x64 and windows server STANDARD 2008 r2 x64 on . I have installed sp3 for Sql 2005 as per the prompts you receive while installing Sql. Database service is installed but not analysis services. I originally
SQL Server SQL Server 2008 version Hi, We have four different software products that use now MS SQL Server 2000 databases. We are using SQL Server 2000 Standard 1 cpu license. I am going to purchase a new server and transfer these software to the new server and replace SQL Server 2000 with version 2005 or 2008
SQL Server Sql Server 200x Stress Tools? I currently have Sql Server 2005 Standard Edition. I have a few legacy Sql Server 2000 Standard Edition databases as well. . . I have implemented the Sql Server 2005 Performance Dashboard Reports as well. Those have helped some
SQL Server SQL Server 2008 Standard and Visual Studio 2008 Professonal I have retailed copies VS 2008 Pro and SQL 2008 Standard. While in VS 2008 I tried to create a SQL database in a ASP.Net 3.51 Web application. I'm getting an error saying I need to install SQL Server 2005 Express Edition. I have all the current patches and both programs. I don't want to down grade my SQL Server. I spent good money on the SQL Server 2008
SQL Server Copy SQL Server 2005 Express database to SQL Server 2000 Hello I have a copy of a SQL Server 2005 Express database - the mdf and .ldf files. If I copy to my
SQL Server SQL Server Database Setup - Almost Newbie First db - - My Form Solicits The Following Information (text box) Photo (upload) OptIn (Checkbox Yes / No) The SQL table CREATION TOOL at my host has the following columns choose: BigInt, Binary, Bit, Char, DataTime, Decimal, Float, Image, Int, Money, NChar, None, NText. Numeric. NVarChar, NVarCharMax, Real, SmallDateTime, SmallInt, SmallMoney looks like you will need a beginner's book on SQL Server to really get started. I can tell you that tinyints estion. Both of these got good ratings on Amazon Beginning SQL Server 2008 for Developers: From Novice to Professional Murach's
SQL Server Where can I find free SQL Server hosting? Hi! Do you know any free MS SQL Server database hosting service? Elton Lika schreef op 14-07-2011 12 09: That's not very likely. Any serious size database is gonna require a version of SQL Server that is beyond SQL Server Express specifications and Express is
SQL Server Sql Server timeout problem Hi all, I am having problems with SQL Server on a clients website. . . I know this email is quite triple checked this is happening) I am connecting to a SQL 2K5 database. Occassionally, I get a timeout whilst querying the DB. It the same time. When it fails, the task manager for SQL goes to 99%. The mem usage is something like 169MB usage drops. However, when I run the same query inside SQL Management studio, it looks like the same thing happens, i