InterWeave Help

Help

How to Use Bulk Load Utility Flows in InterWeave Integration Manager

How to Use Bulk Load Utility Flows in InterWeave Integration Manager

NOTE: Turn off Scheduled flows before starting Utility flows!!!

UTILITY FLOW DEFINITION

There are 2 types of Utility flows:

  1. Bulk Load flows (with DR or DRS at the end of the flow name) DR = Date Range or DRS = Date Range & Size (Check radio button “Scheduled”)- for Bulk Load by period (Date Range) and batch size (S).
  2. Single transaction flows (with N or NF at the end of the flow name) N = Name/Number (Check radio button “Single Run”) – for one time particular transaction sync or testing (Account/Opportunity/Product/Quote… etc.)

In order to use Utility flows you will need to:

  1. Login into your ISP (Interweave Integration Solution) flows manager at (http://iw0.interweave.biz:8080/iw-business-daemon/IWLogin.jsp) – just type in your username and password and click Login button.
  2. Run one of Utility Flows (second low section of the screen) to test flow.

Please refer to this video for details: http://www.youtube.com/watch?v=S7crX_OEEpc&feature=channel&list=UL

(Production and Utilities Flows are show on 4 min of the video)

HOW TO RUN BULK LOAD FLOWS

Bulk load flows are marked with DR or DRS at the end of flow’s name.

Flows to use for SF to QB direction: (all flows are DR type except Account flow)

    For Customers:                 SFAcct2QBCustDRS

    For Items:                           SFProd2QBItemDR

    For Invoices:                      SFAcctOpp2QBInvoicesDR

    For Sales Orders:              SFOpp2QBSODR

    For Sales Receipts:          SFOpp2QBSRDR

    For Vendors:                      SFAcct2QBVendorDR

    For Purchase Orders:     SFOpp2QBPODR

Flows to use for QB to SF direction: (all flows are DRS type)

    For Customers:                 QBCust2SFAcctDRS

    For Items:                           QBItem2SFProdDRS

    For Invoices:                      QBInvoices2SFAcctOppDRS

    For Sales Orders:              QBSO2SFAcctOppDRS

    For Sales Receipts:          QBSR2SFAcctOppDRS

    For Vendors:                      QBVend2SFAcctDRS

    For Purchase Orders:     QBPR2SFAcctOppDRS

IN ORDER TO RUN UTILITY FLOWS YOU WILL NEED TO:

  1. Login into your ISP (InterWeave Solution Portal) Integration Flows Manager at (https://iw0.interweave.biz:8443/iw-business-daemon/IWLogin.jsp) – just type in your username and password and click Login button.
  2. Stop any started scheduled flow(s)
  3. Click on the flow you want to do a bulk load with.
  4. Fill in 3 opened fields below Utility Flows section
    1. BatchSize
    1. EndDateTime
    1. StartDateTime
  5. Set Counter of the flow (second column from the right) to 0
  6. Check Start Checkbox (second column from the left) and auto-refresh checkbox (right top corner of the page – it will refresh manager’s page every 5 seconds)
  7. Click Submit button to run flow
  8. Monitor flow results in a log (the log is accessible by clicking on highlighted number/number in Runs column for this flow)
  9. Restart scheduled flow(s) if needed after bulk load is completed
  10. Logout from Integration Manager

Please refer to this video for details: http://www.youtube.com/watch?v=S7crX_OEEpc&feature=channel&list=UL

(Production and Utilities Flows are show on 4 min of the video)

All DRS flows have 3 parameters:

BATCH SIZE:

Recommended values for QB to SF direction:

  1. For Customers: QBCust2SFAcctDRS = 4 if customer contacts also are loaded from QB to SF; 20 otherwise
  2. For Items: QBItem2SFProdDRS = 25
  3. For Invoices: QBInvoices2SFAcctOppDRS = 4 (with line items) or 8-24 otherwise

For SF to QB direction batch size is controlled by SF. It is recommended to use 100 for any flow.

INTERVAL:

StartDateTime, EndDateTime – interval that is supposed to be loaded.

Formats:

  1. YYYY-MM-DD – created date for Customers and Items, transaction date for Invoices – (use for history data bulk load)
  2. YYYY-MM-DD HH:MM:SS – last modified date/time for all types. 24 hours clock should be used – (use to catch up with current missing data)* – for unlimited period (QB to SF only!). Could be used in Start and End DateTime or in just one of them (Do not supported for transactions – Invoices, Sales Receipts, Sales Orders etc.!)
  3. SF to QB direction Date/Time format (UTC): 2022-01-30T00:00:00.0Z (YYYY-MM-DDTHH:MM:SS)
  4. QB to SF direction Date/Time format: 2022-01-30 00:00:00 (YYYY-MM-DD HH:MM:SS)
  5. For “QuickBooks Online” QB to SF direction Date format: 2022-01-30 (YYYY-MM-DD)

INTERVAL NOTE FOR INVOICE FLOW ONLY:

  1. If you put to StartDate and/or EndDate just date (YYYY-MM-DD) – Opportunity CloseDate is used
  2. If you put to StartDate and/or EndDate timestamp (YYYY-MM-DDTHH:MM:SS.MZ) and BatchSize is empty – LastModifiedDate is used
  3. If you put to StartDate and/or EndDate timestamp (YYYY-MM-DDTHH:MM:SS.MZ) and BatchSize is not empty – CreatedDate is used

EXAMPLES:

  • EndDateTime=YYYY-MM-DD (or YYYY-MM-DD HH:MM:SS) – to load all records set something in the future (e.g. 2022-01-01 00:00:00) or exact date and time for specific period (e.g. 2012-10-01 00:00:00)
  • StartDateTime=YYYY-MM-DD (or YYYY-MM-DD HH:MM:SS) – to load all records set something in the past (e.g. 2000-01-01 00:00:00) or exact date and time for specific period (e.g. 2012-01-21 00:00:00)

When each of those flows is started first time – Counter (second column from the right) MUST be equal to 0.

If you want to stop bulk load – just remember all 3 parameters and Counter to resume later.

It is recommended to run bulk loads after hours or on weekend.

GENERAL NOTES:

  1. SF to QB and QB to SF direction flows for any type of bulk load require to set Permitted sync Operation to Create Only in Profile (temporary change)
  2. SF to QB direction flows require to set Permitted sync Operation for opposite direction to None in Profile (temporal change)
  3. QB to SF direction flows do not requires to set Permitted sync Operation for opposite direction to None (no change in Profile)
  4. Before flow is started, it is beneficial to have auto-refresh checkbox checked – this will refresh manager’s page every 5 seconds so you can monitor the progress of the flow. The Counter column will show you the number of already finished runs – every run is equal a number of transactions defined by BatchSize. It will stop when there are no more records to process.
  5. Every new bulk load requires Counter (second column from the right) to be reset to 0.
  6. Counter shows how many batches of size “batch size” are loaded.
  7. If you want to stop a particular load and then start from the same place – you need just to restore the Counter to the value you’ve stopped the load with.
  8. Remember that utility flows parameters are lost after logging out.
  9. If you bulk loading QB Customers/SF Accounts with Contacts then BatchSize cannot be more than 4 otherwise you will get an LIMIT_EXCEEDED error:
  10. “Query executed; Result= sf:LIMIT_EXCEEDEDLIMIT_EXCEEDED: Cannot have more than 10 chunks in a single operation. Please rearrange the data to reduce chunking.LIMIT_EXCEEDED Cannot have more than 10 chunks in a single operation. Please rearrange the data to reduce chunking.”
  11. If you restarting any SF to QB DR/DRS flow – you need to clear Return String! (Return String is a Property on top of the batch size in the lower frame. It is not there when you first time configuring flow (setting up Batch Size, Start Date and End Date). It appears after first run and contains SF Query Locator. If you need to restart DRS flow – you need to clear it or logout and login.

We keep their request locator from Salesforce but they (SF) have a mechanism that expires session after some timeout period. Using the old query locator is causing “Query Locator” error. This error is very rare. In any case we will also put a locking mechanism in the infinite run of the flow after such error.

Leave a Reply

Your email address will not be published. Required fields are marked *