Batch Processing

The ADO.NET Provider for TIBCO(R) Data Virtualization enables you to take advantage of the bulk load support in TDV through CompositeDataAdapters. You can use the Batch API to execute related SQL data manipulation statements simultaneously.

Using the ADO.NET Batch API

Performing a batch update consists of the following basic steps:

1.

Define custom parameterized SQL statements in CompositeCommand objects.

2.

Set the UpdatedRowSource property of the CompositeCommand object to "UpdateRowSource.None".

3.

Assign the CompositeCommand objects to the CompositeDataAdapter.

4.

Add the parameters to the command.

5.

Call the CompositeDataAdapter's Update method. Pass in a DataSet or DataTable containing your changes.

Controlling Batch Size

Depending on factors such as the size of the request, your network resources, and the performance of the server, you may gain performance by executing several smaller batch requests. You can control the size of each batch by setting the CompositeDataAdapter's UpdateBatchSize property to a positive integer.

Bulk Insert

The following code prepares a single batch that inserts records in bulk and retrieves the new records' Ids. The example executes a batch insert of new DataRows, which have the "Added" state.

C#

CompositeDataAdapter adapter = new CompositeDataAdapter();

using (CompositeConnection conn = new CompositeConnection("Host=myHost;Domain=myDomain;DataSource=myDataSource;User=myUser;Password=myPassword")) {

  conn.Open();

  adapter.InsertCommand = conn.CreateCommand();

  adapter.InsertCommand.CommandText = "INSERT INTO [Public].[Sample].Products (ProductName) VALUES (@ProductName)";

  adapter.InsertCommand.UpdatedRowSource = UpdateRowSource.None;

  adapter.InsertCommand.Parameters.Add("@ProductName", "ProductName");

  DataTable batchDataTable = new DataTable();

  batchDataTable.Columns.Add("ProductName", typeof(string));

  batchDataTable.Rows.Add("Ikura");

  batchDataTable.Rows.Add("Konbu");

  adapter.UpdateBatchSize = 2;

  adapter.Update(batchDataTable);

  CompositeCommand cmd = new CompositeCommand("SELECT * FROM LastResultInfo#TEMP", conn);

  adapter = new CompositeDataAdapter(cmd);

  DataTable res = new DataTable();

  adapter.Fill(res);

  foreach (DataRow row in res.Rows)

    foreach(DataColumn col in res.Columns)

      Console.Write("{0}: {1}", col.ColumnName, row[col]);

}

VB.NET

Dim adapter As New CompositeDataAdapter()

Using conn As New CompositeConnection("Host=myHost;Domain=myDomain;DataSource=myDataSource;User=myUser;Password=myPassword")

  conn.Open()

  adapter.InsertCommand = conn.CreateCommand()

  adapter.InsertCommand.CommandText = "INSERT INTO [Public].[Sample].Products (Id) VALUES (@ProductName)"

  adapter.InsertCommand.UpdatedRowSource = UpdateRowSource.None

  adapter.InsertCommand.Parameters.Add("@ProductName", "ProductName")

  Dim batchDataTable As New DataTable()

  batchDataTable.Columns.Add("ProductName", GetType(String))

  batchDataTable.Rows.Add("Konbu")

  batchDataTable.Rows.Add("Ikura")

  adapter.UpdateBatchSize = 2

  adapter.Update(batchDataTable)

  Dim cmd As New CompositeCommand("SELECT * FROM LastResultInfo#TEMP", conn)

  adapter = New CompositeDataAdapter(cmd)

  Dim res As New DataTable()

  adapter.Fill(res)

  For Each row As DataRow In res.Rows

    For Each col As DataColumn In res.Columns

      Console.WriteLine("{0}: {1}", col.ColumnName, row(col))

    Next

  Next

End Using

Bulk Update

A batch update additionally requires the primary key of each row to update. The following example executes a batch for all DataRow records with a "Modified" state:

C#

CompositeDataAdapter adapter = new CompositeDataAdapter();

using (CompositeConnection conn = new CompositeConnection("Host=myHost;Domain=myDomain;DataSource=myDataSource;User=myUser;Password=myPassword")) {

  conn.Open();

  adapter.UpdateCommand = conn.CreateCommand();

  adapter.UpdateCommand.CommandText = "UPDATE [Public].[Sample].Products SET ProductName=@ProductName WHERE Id=@Id";

  adapter.UpdateCommand.Parameters.Add("@ProductName", "ProductName");

  adapter.UpdateCommand.Parameters.Add("@Id", "Id");

  adapter.UpdateCommand.UpdatedRowSource = UpdateRowSource.None;

  adapter.UpdateBatchSize = 2;

  adapter.Update(dataTable);

}

VB.NET

Dim adapter As New CompositeDataAdapter()

Using conn As New CompositeConnection("Host=myHost;Domain=myDomain;DataSource=myDataSource;User=myUser;Password=myPassword")

  conn.Open()

  adapter.UpdateCommand = conn.CreateCommand()

  adapter.UpdateCommand.CommandText = "UPDATE [Public].[Sample].Products SET ProductName=@ProductName WHERE Id=@Id"

  adapter.UpdateCommand.Parameters.Add("@ProductName", "ProductName")

  adapter.UpdateCommand.Parameters.Add("@Id", "Id")

  adapter.UpdateCommand.UpdatedRowSource = UpdateRowSource.None

  adapter.UpdateBatchSize = 2

  adapter.Update(dataTable)

End Using

Bulk Delete

The following code prepares a single batch that deletes records in bulk. The primary key for each row is required. The following example executes a batch for all DataRow records with a "Deleted" state:

C#

CompositeDataAdapter adapter = new CompositeDataAdapter();

using (CompositeConnection conn = new CompositeConnection("Host=myHost;Domain=myDomain;DataSource=myDataSource;User=myUser;Password=myPassword")) {

  conn.Open();

  adapter.DeleteCommand = conn.CreateCommand();

  adapter.DeleteCommand.CommandText = "DELETE FROM [Public].[Sample].Products WHERE Id=@Id";

  adapter.DeleteCommand.Parameters.Add("@Id", "Id");

  adapter.DeleteCommand.UpdatedRowSource = UpdateRowSource.None;

  adapter.UpdateBatchSize = 2;

  adpater.Update(table);

}

VB.NET

Dim adapter As New CompositeDataAdapter()

Using conn As New CompositeConnection("Host=myHost;Domain=myDomain;DataSource=myDataSource;User=myUser;Password=myPassword")

  conn.Open()

  adapter.DeleteCommand = conn.CreateCommand()

  adapter.DeleteCommand.CommandText = "DELETE FROM [Public].[Sample].Products WHERE Id=@Id"

  adapter.DeleteCommand.Parameters.Add("@Id", "Id")

  adapter.DeleteCommand.UpdatedRowSource = UpdateRowSource.None

  adapter.UpdateBatchSize = 2

  adpater.Update(table)

End Using