APPLIES TO: SQL Server (starting with 2008) Azure SQL Database Azure SQL Data Warehouse Parallel Data Warehouse
-->
Imports a data file into a database table or view in a user-specified format in SQL Server
SyntaxArguments
database_name
Is the database name in which the specified table or view resides. If not specified, this is the current database.
schema_name
Is the name of the table or view schema. schema_name is optional if the default schema for the user performing the bulk-import operation is schema of the specified table or view. If schema is not specified and the default schema of the user performing the bulk-import operation is different from the specified table or view, SQL Server returns an error message, and the bulk-import operation is canceled.
table_name
Is the name of the table or view to bulk import data into. Only views in which all columns refer to the same base table can be used. For more information about the restrictions for loading data into views, see INSERT (Transact-SQL).
'data_file'
Is the full path of the data file that contains data to import into the specified table or view. BULK INSERT can import data from a disk (including network, floppy disk, hard disk, and so on).
data_file must specify a valid path from the server on which SQL Server is running. If data_file is a remote file, specify the Universal Naming Convention (UNC) name. A UNC name has the form SystemnameShareNamePathFileName. For example:
Applies to: SQL Server 2017 (14.x) CTP 1.1.
Beginning with SQL Server 2017 (14.x) CTP1.1, the data_file can be in Azure blob storage. In that case, you need to specify data_source_name option.
Important
Azure SQL Database does not support reading from Windows files.
'data_source_name'
Applies to: SQL Server 2017 (14.x) CTP 1.1. Is a named external data source pointing to the Azure Blob storage location of the file that will be imported. The external data source must be created using the TYPE = BLOB_STORAGE option added in SQL Server 2017 (14.x) CTP 1.1. For more information, see CREATE EXTERNAL DATA SOURCE.
BATCHSIZE =batch_size
Specifies the number of rows in a batch. Each batch is copied to the server as one transaction. If this fails, SQL Server commits or rolls back the transaction for every batch. By default, all data in the specified data file is one batch. For information about performance considerations, see 'Remarks,' later in this topic.
CHECK_CONSTRAINTS
Specifies that all constraints on the target table or view must be checked during the bulk-import operation. Without the CHECK_CONSTRAINTS option, any CHECK and FOREIGN KEY constraints are ignored, and after the operation, the constraint on the table is marked as not-trusted.
Note
UNIQUE, and PRIMARY KEY constraints are always enforced. When importing into a character column that is defined with a NOT NULL constraint, BULK INSERT inserts a blank string when there is no value in the text file.
At some point, you must examine the constraints on the whole table. If the table was non-empty before the bulk-import operation, the cost of revalidating the constraint may exceed the cost of applying CHECK constraints to the incremental data.
A situation in which you might want constraints disabled (the default behavior) is if the input data contains rows that violate constraints. With CHECK constraints disabled, you can import the data and then use Transact-SQL statements to remove the invalid data.
Note
The MAXERRORS option does not apply to constraint checking.
CODEPAGE = { 'ACP' | 'OEM' | 'RAW' | 'code_page' }
Specifies the code page of the data in the data file. CODEPAGE is relevant only if the data contains char, varchar, or text columns with character values greater than 127 or less than 32.
Important
CODEPAGE is not a supported option on Linux for SQL Server 2017 (14.x). For SQL Server 2019 preview, only the 'RAW' option is allowed for CODEPAGE.
Note
Microsoft recommends that you specify a collation name for each column in a format file.
DATAFILETYPE = { 'char' | 'native' | 'widechar' | 'widenative' }
Specifies that BULK INSERT performs the import operation using the specified
The error file is created when the command is executed. An error occurs if the file already exists. Additionally, a control file that has the extension .ERROR.txt is created. This references each row in the error file and provides error diagnostics. As soon as the errors have been corrected, the data can be loaded.
Applies to: SQL Server 2017 (14.x) CTP 1.1.Beginning with SQL Server 2017 (14.x), the error_file_path can be in Azure blob storage.
'errorfile_data_source_name'
Applies to: SQL Server 2017 (14.x) CTP 1.1.Is a named external data source pointing to the Azure Blob storage location of the error file that will contain errors found during the import. The external data source must be created using the TYPE = BLOB_STORAGE option added in SQL Server 2017 (14.x) CTP 1.1. For more information, see CREATE EXTERNAL DATA SOURCE.
FIRSTROW =first_row
Specifies the number of the first row to load. The default is the first row in the specified data file. FIRSTROW is 1-based.
Note
The FIRSTROW attribute is not intended to skip column headers. Skipping headers is not supported by the BULK INSERT statement. When skipping rows, the SQL Server Database Engine looks only at the field terminators, and does not validate the data in the fields of skipped rows.
FIRE_TRIGGERS
Specifies that any insert triggers defined on the destination table execute during the bulk-import operation. If triggers are defined for INSERT operations on the target table, they are fired for every completed batch.
If FIRE_TRIGGERS is not specified, no insert triggers execute.
FORMATFILE_DATASOURCE = 'data_source_name'
Applies to: SQL Server 2017 (14.x) 1.1. Is a named external data source pointing to the Azure Blob storage location of the format file that will define the schema of imported data. The external data source must be created using the TYPE = BLOB_STORAGE option added in SQL Server 2017 (14.x) CTP 1.1. For more information, see CREATE EXTERNAL DATA SOURCE.
KEEPIDENTITY
Specifies that identity value or values in the imported data file are to be used for the identity column. If KEEPIDENTITY is not specified, the identity values for this column are verified but not imported and SQL Server automatically assigns unique values based on the seed and increment values specified during table creation. If the data file does not contain values for the identity column in the table or view, use a format file to specify that the identity column in the table or view is to be skipped when importing data; SQL Server automatically assigns unique values for the column. For more information, see DBCC CHECKIDENT (Transact-SQL).
For more information, see about keeping identify values see Keep Identity Values When Bulk Importing Data (SQL Server).
KEEPNULLS
Specifies that empty columns should retain a null value during the bulk-import operation, instead of having any default values for the columns inserted. For more information, see Keep Nulls or Use Default Values During Bulk Import (SQL Server).
KILOBYTES_PER_BATCH =kilobytes_per_batch
Specifies the approximate number of kilobytes (KB) of data per batch as kilobytes_per_batch. By default, KILOBYTES_PER_BATCH is unknown. For information about performance considerations, see 'Remarks,' later in this topic.
LASTROW =last_rowSpecifies the number of the last row to load. The default is 0, which indicates the last row in the specified data file.
MAXERRORS =max_errors
Specifies the maximum number of syntax errors allowed in the data before the bulk-import operation is canceled. Each row that cannot be imported by the bulk-import operation is ignored and counted as one error. If max_errors is not specified, the default is 10.
Note
The MAX_ERRORS option does not apply to constraint checks or to converting money and bigint data types.
ORDER ( { column [ ASC | DESC ] } [ ,.. n ] )
Specifies how the data in the data file is sorted. Bulk import performance is improved if the data being imported is sorted according to the clustered index on the table, if any. If the data file is sorted in a different order, that is other than the order of a clustered index key or if there is no clustered index on the table, the ORDER clause is ignored. The column names supplied must be valid column names in the destination table. By default, the bulk insert operation assumes the data file is unordered. For optimized bulk import, SQL Server also validates that the imported data is sorted.
n
Is a placeholder that indicates that multiple columns can be specified.
ROWS_PER_BATCH =rows_per_batch
Indicates the approximate number of rows of data in the data file.
By default, all the data in the data file is sent to the server as a single transaction, and the number of rows in the batch is unknown to the query optimizer. If you specify ROWS_PER_BATCH (with a value > 0) the server uses this value to optimize the bulk-import operation. The value specified for ROWS_PER_BATCH should approximately the same as the actual number of rows. For information about performance considerations, see 'Remarks,' later in this topic.
TABLOCK
Specifies that a table-level lock is acquired for the duration of the bulk-import operation. A table can be loaded concurrently by multiple clients if the table has no indexes and TABLOCK is specified. By default, locking behavior is determined by the table option table lock on bulk load. Holding a lock for the duration of the bulk-import operation reduces lock contention on the table, in some cases can significantly improve performance. For information about performance considerations, see 'Remarks,' later in this topic.
For columnstore index. the locking behaviour is different because it is internally divided into multiple rowsets. Each thread loads data exclusively into each rowset by taking a X lock on the rowset allowing parallel data load with concurrent data load sessions. The use of TABLOCK option will cause thread to take an X lock on the table (unlike BU lock for traditional rowsets) which will prevent other concurrent threads to load data concurrently.
Input file format options
FORMAT = 'CSV'
Applies to: SQL Server 2017 (14.x) CTP 1.1. Specifies a comma separated values file compliant to the RFC 4180 standard.
FIELDQUOTE = 'field_quote'
Applies to: SQL Server 2017 (14.x) CTP 1.1. Specifies a character that will be used as the quote character in the CSV file. If not specified, the quote character (') will be used as the quote character as defined in the RFC 4180 standard.
FORMATFILE = 'format_file_path'
Specifies the full path of a format file. A format file describes the data file that contains stored responses created by using the bcp utility on the same table or view. The format file should be used if:
Applies to: SQL Server 2017 (14.x) CTP 1.1.
Beginning with SQL Server 2017 (14.x) CTP 1.1, the format_file_path can be in Azure blob storage.
FIELDTERMINATOR ='field_terminator'
Specifies the field terminator to be used for char and widechar data files. The default field terminator is t (tab character). For more information, see Specify Field and Row Terminators (SQL Server).
ROWTERMINATOR ='row_terminator'
Specifies the row terminator to be used for char and widechar data files. The default row terminator is rn (newline character). For more information, see Specify Field and Row Terminators (SQL Server). Compatibility
BULK INSERT enforces strict data validation and data checks of data read from a file that could cause existing scripts to fail when they are executed on invalid data. For example, BULK INSERT verifies that:
Data TypesString-to-Decimal Data Type Conversions
The string-to-decimal data type conversions used in BULK INSERT follow the same rules as the Transact-SQL CONVERT function, which rejects strings representing numeric values that use scientific notation. Therefore, BULK INSERT treats such strings as invalid values and reports conversion errors.
To work around this behavior, use a format file to bulk import scientific notation float data into a decimal column. In the format file, explicitly describe the column as real or float data. For more information about these data types, see float and real (Transact-SQL).
Note
Format files represent real data as the SQLFLT4 data type and float data as the SQLFLT8 data type. For information about non-XML format files, see Specify File Storage Type by Using bcp (SQL Server).
Example of Importing a Numeric Value that Uses Scientific Notation
This example uses the following table:
The user wants to bulk import data into the
t_float table. The data file, C:t_float-c.dat, contains scientific notation float data; for example:
However, BULK INSERT cannot import this data directly into
t_float , because its second column, c2 , uses the decimal data type. Therefore, a format file is necessary. The format file must map the scientific notation float data to the decimal format of column c2 .
The following format file uses the
SQLFLT8 data type to map the second data field to the second column:
To use this format file (using the file name
C:t_floatformat-c-xml.xml ) to import the test data into the test table, issue the following Transact-SQL statement:
Important
Azure SQL Database does not support reading from Windows files.
Data Types for Bulk Exporting or Importing SQLXML Documents
To bulk export or import SQLXML data, use one of the following data types in your format file:
General Remarks
For a comparison of the BULK INSERT statement, the INSERT .. SELECT * FROM OPENROWSET(BULK..) statement, and the bcp command, see Bulk Import and Export of Data (SQL Server).
For information about preparing data for bulk import, see Prepare Data for Bulk Export or Import (SQL Server).
The BULK INSERT statement can be executed within a user-defined transaction to import data into a table or view. Optionally, to use multiple matches for bulk importing data, a transaction can specify the BATCHSIZE clause in the BULK INSERT statement. If a multiple-batch transaction is rolled back, every batch that the transaction has sent to SQL Server is rolled back.
InteroperabilityImporting Data from a CSV file
Beginning with SQL Server 2017 (14.x) CTP 1.1, BULK INSERT supports the CSV format.
Before SQL Server 2017 (14.x) CTP 1.1, comma-separated value (CSV) files are not supported by SQL Server bulk-import operations. However, in some cases, a CSV file can be used as the data file for a bulk import of data into SQL Server. For information about the requirements for importing data from a CSV data file, see Prepare Data for Bulk Export or Import (SQL Server). Logging Behavior
For information about when row-insert operations that are performed by bulk import are logged in the transaction log, see Prerequisites for Minimal Logging in Bulk Import.
Restrictions
When using a format file with BULK INSERT, you can specify up to 1024 fields only. This is same as the maximum number of columns allowed in a table. If you use BULK INSERT with a data file that contains more than 1024 fields, BULK INSERT generates the 4822 error. The bcp utility does not have this limitation, so for data files that contain more than 1024 fields, use the bcp command.
Performance Considerations
If the number of pages to be flushed in a single batch exceeds an internal threshold, a full scan of the buffer pool might occur to identify which pages to flush when the batch commits. This full scan can hurt bulk-import performance. A likely case of exceeding the internal threshold occurs when a large buffer pool is combined with a slow I/O subsystem. To avoid buffer overflows on large machines, either do not use the TABLOCK hint (which will remove the bulk optimizations) or use a smaller batch size (which preserves the bulk optimizations).
Because computers vary, we recommend that you test various batch sizes with your data load to find out what works best for you.
SecuritySecurity Account Delegation (Impersonation)
If a user uses a SQL Server login, the security profile of the SQL Server process account is used. A login using SQL Server authentication cannot be authenticated outside of the Database Engine. Therefore, when a BULK INSERT command is initiated by a login using SQL Server authentication, the connection to the data is made using the security context of the SQL Server process account (the account used by the SQL Server Database Engine service). To successfully read the source data you must grant the account used by the SQL Server Database Engine, access to the source data.In contrast, if a SQL Server user logs on by using Windows Authentication, the user can read only those files that can be accessed by the user account, regardless of the security profile of the SQL Server process.
When executing the BULK INSERT statement by using sqlcmd or osql, from one computer, inserting data into SQL Server on a second computer, and specifying a data_file on third computer by using a UNC path, you may receive a 4861 error.
To resolve this error, use SQL Server Authentication and specify a SQL Server login that uses the security profile of the SQL Server process account, or configure Windows to enable security account delegation. For information about how to enable a user account to be trusted for delegation, see Windows Help.
For more information about this and other security considerations for using BULK INSERT, see Import Bulk Data by Using BULK INSERT or OPENROWSET(BULK..) (SQL Server).
Permissions
Requires INSERT and ADMINISTER BULK OPERATIONS permissions. In Azure SQL Database, INSERT and ADMINISTER DATABASE BULK OPERATIONS permissions are required. Additionally, ALTER TABLE permission is required if one or more of the following is true:
ExamplesA. Using pipes to import data from a file
The following example imports order detail information into the
AdventureWorks2012.Sales.SalesOrderDetail table from the specified data file by using a pipe (| ) as the field terminator and |n as the row terminator.
Important
Azure SQL Database does not support reading from Windows files.
B. Using the FIRE_TRIGGERS argument
The following example specifies the
FIRE_TRIGGERS argument.
Important
Azure SQL Database does not support reading from Windows files.
C. Using line feed as a row terminator
The following example imports a file that uses the line feed as a row terminator such as a UNIX output:
Note
Due to how Microsoft Windows treats text files (n automatically gets replaced with rn).
Important
Azure SQL Database does not support reading from Windows files.
D. Specifying a code page
The following example show how to specify a code page.
Important
Azure SQL Database does not support reading from Windows files.
E. Importing data from a CSV file
The following example show how to specify a CSV file, skipping the header (first row), using
; as field terminator and 0x0a as line terminator:
Important
Azure SQL Database does not support reading from Windows files.
F. Importing data from a file in Azure blob storage
The following example shows how to load data from a csv file in an Azure blob storage location, which has been configured as an external data source. This requires a database scoped credential using a shared access signature.
![]()
Important
Azure SQL Database does not support reading from Windows files.
G. Importing data from a file in Azure blob storage and specifying an error file
The following example shows how to load data from a csv file in an Azure blob storage location, which has been configured as an external data source and also specifying an error file. This requires a database scoped credential using a shared access signature. Note that if running on Azure SQL Database, ERRORFILE option should be accompanied by ERRORFILE_DATA_SOURCE otherwise the import might fail with permissions error. The file specified in ERRORFILE should not exist in the container.
For complete
BULK INSERT examples including configuring the credential and external data source, see Examples of Bulk Access to Data in Azure Blob Storage.
Additional Examples
Other
BULK INSERT examples are provided in the following topics:
See Also
Bulk Import and Export of Data (SQL Server)
bcp Utility Format Files for Importing or Exporting Data (SQL Server) INSERT (Transact-SQL) OPENROWSET (Transact-SQL) Prepare Data for Bulk Export or Import (SQL Server) sp_tableoption (Transact-SQL)
I would like to load employee photos into the table per_images. Per_images used to have a long raw column for the image, but this has changed to a blob.
The following sql loader control file used to work with the long raw column:
where 9529 is the size of the jpg picture. How should I change the raw(9529) to instead allow loading into a blob column?
SuperdooperheroSuperdooperhero
How To Load Pic X (1 Data Type Using Sql Loader Tool
2,4181414 gold badges4646 silver badges8989 bronze badges
1 Answer
I usually use a different way to load BLOB data using SQL*Loader. Basically I import a text file that contains the filenames and in the control file, I then tell SQL*Loader that the actual content comes from a lobfile.
In your case this means you would need to create text file that contains (only) the filename of the jpg. The control file then should look like this:
The input file
data.txt would then look like this:
It's important that the
image lobfile .. part is at the end and that any constant definition comes first in the control file.
Using this kind of approach seems much simpler to me because you do not need to know the size of the input file and you can load more than one picture with a SQL*Loader run which is most probably a lot faster if you need to load a large amount of pictures.
If you want to load more than one picture, the input file needs to contain the constant values that you supplied so far inside the control file. Take the following input file:
Then you can load all three pictures with a single control file:
The control file will never change, only the contents of the
data.txt file.
Your original control file works for me, if the a_horse_with_no_namea_horse_with_no_name
raw(9529) is removed completely: Homeworld remastered guide.
319k5050 gold badges491491 silver badges594594 bronze badges
Got a question that you canât ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Not the answer you're looking for? Browse other questions tagged oracleblobsql-loader or ask your own question.
The problem is, Sql Loader is terminated on 'ACC_SORT_CD'.I am facing the problem in every column after specifying DATE datatype.It is not able to read the column which is coming after DATE datatype.
I have tried with another file as well but same problem is present.Please let me know what I am missing?
Bob Jarvis
36.4k55 gold badges6262 silver badges8888 bronze badges
ArzuArzu
Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.
Browse other questions tagged oraclesql-loader or ask your own question.
I was loading data using SQL*Loader and when making the control file I used the table definition and accidentally left the INTEGER data type on the 'version' line.
And in the 'version' field (data type integer) it inserted the value 808594481.
I'm having a hard time understanding how it processed this value -- I'm assuming it took it as a literal .. but is that the sum of the ASCII representations of each letter?
NOPE!
returns 666 (which, btw is hilarious).
concatenate ascii values?
returns 737884697169716982
I'm hoping someone out there knows the answer.
This is the actual control file:
Table DDL:
Data:
a_horse_with_no_name
319k5050 gold badges491491 silver badges594594 bronze badges
Robert ClaytonRobert Clayton
1 Answer
If you split 808594481 into bytes as it would be encoded in a 32 bit twos complement encoding, and treat each byte as an ascii-encoded character, you get '02,1' or '1,20' depending on byte order. You probably inserted a string that starts or ends with one of those, and some layer between your code and the database silently converted it to an integer.
recursiverecursive
63.8k2222 gold badges122122 silver badges211211 bronze badges
Got a question that you canât ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Not the answer you're looking for? Browse other questions tagged oraclesql-loaderctl or ask your own question.Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |