[IQUG] IQ 16 loads to very wide tables

Baker, Chris c.baker at sap.com
Thu Dec 20 14:06:00 MST 2018


The packet size is the default =- 7300 bytes.  You can take it up to 64KB by starting the server with the -p parameter (e.g. -p 65300).

Chris Baker | Database Engineering Evangelist | CERT | PI HANA Platform Data Management | SAP
T +1 416-226-7033<tel:+1%20416-226-7033> | M +1 647-224-2033<tel:+1%20647-224-2033> | TF +1 866-716-8860<tel:+1%20866-716-8860>
SAP Canada Inc. 445 Wes Graham Way, Waterloo, N2L 6R2<x-apple-data-detectors://17/1>
c.baker at sap.com<mailto:c.baker at sap.com> | www.sap.com<http://www.sap.com/>

https://sap.na.pgiconnect.com/I826572
Conference tel: 1-866-312-7353,,9648565377#<tel:1-866-312-7353,,9648565377%23>

From: Antonian, Vahe <vahe.antonian.2 at credit-suisse.com>
Sent: Thursday, December 20, 2018 3:54 PM
To: Baker, Chris <c.baker at sap.com>; 'iqug at iqug.org' <iqug at iqug.org>
Subject: RE: IQ 16 loads to very wide tables


  *   Around 570K records in the data file (5GB is the file size)
  *   No pipes are used to my knowledge anywhere
  *   SQL files called with dbisql command and connection terminates after 570K rows loaded to table
  *   100K rows were successfully loaded from client side with NO Temp Buffer errors.
  *   P1 Case 555268/2018  registered with SAP
  *   Network packet size
     *   $ dbisql -c "uid=DBA;pwd=xxxxxxxx;dbn=UNYDS31900_IQ;eng=UNYDS31900_IQ" -nogui
     *
     *   (DBA)> select connection_property ('PacketSize')
     *   connection_property('PacketSize')
     *   ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
     *   7300


Vahe Antonian
Database IB Ops Service
+1 212 538 1642 (*106 1642)
From: Baker, Chris <c.baker at sap.com<mailto:c.baker at sap.com>>
Sent: Thursday, December 20, 2018 12:06 PM
To: Antonian, Vahe (MLEA 42); 'iqug at iqug.org'
Subject: RE: IQ 16 loads to very wide tables

How many rows are in the source data?  How large is the source file?

The data needs to be sent from client to server, so probably has to be materialized somewhere on the server first.  I am not sure about the implementation (i.e. is a pipe used? /tmp?) but somewhere you will probably run out of space (yes, temp cache might be used here) if it is not cleaned up.

Are you also issuing an explicit 'commit' after a client-side load?  Perhaps the data needs to be retained in case of a rollback, so that is why you might see temp cache usage growing until a 'commit' is issued on the connection.

What happens if you cut down the number of rows in a client-side load.  Does the failure still occur?

You probably need to open a support incident on this.

Although it is probably not relevant, what is the server packet size?  A larger packet size might simple help with getting the data to the server faster.

Chris

Chris Baker | Database Engineering Evangelist | CERT | PI HANA Platform Data Management | SAP
T +1 416-226-7033<tel:+1%20416-226-7033> | M +1 647-224-2033<tel:+1%20647-224-2033> | TF +1 866-716-8860<tel:+1%20866-716-8860>
SAP Canada Inc. 445 Wes Graham Way, Waterloo, N2L 6R2<x-apple-data-detectors://17/1>
c.baker at sap.com<mailto:c.baker at sap.com> | www.sap.com<http://www.sap.com/>

https://sap.na.pgiconnect.com/I826572
Conference tel: 1-866-312-7353,,9648565377#<tel:1-866-312-7353,,9648565377%23>

From: iqug-bounces at iqug.org<mailto:iqug-bounces at iqug.org> <iqug-bounces at iqug.org<mailto:iqug-bounces at iqug.org>> On Behalf Of Antonian, Vahe
Sent: Thursday, December 20, 2018 11:48 AM
To: 'iqug at iqug.org' <iqug at iqug.org<mailto:iqug at iqug.org>>
Subject: [IQUG] IQ 16 loads to very wide tables


Struggling with IQ 16 client side loads to a very wide table with 7000+ columns. After enough executions, it eventually starts to fails with temp cache errors. Interestingly, the problem never happens with server side loads and it never happens after a bounce of the instance.

temp Bufman: All buffer cache pages are in use

We think it's a problem flushing out pages from the cache b/c it never happens on the first few loads after a bounce but eventually starts happening after enough loads and activity has run. Temp cache is hugely oversized and SP11.21 was recently applied as per advice from the SAP engineer to address a few known bugs.

Just curious if anyone else has any tips or tricks for IQ16 loads to very wide tables that might help us isolate what's happening?  The process worked fine on IQ 15

Thanks,

Vahe Antonian
CREDIT SUISSE SERVICES (USA) LLC
Credit Suisse Services(USA)LLC | Database IB Ops Service, MLEA 42
Eleven Madison Avenue | New York NY 10010-3629 | Americas
Phone +1 212 538 1642
vahe.antonian.2 at credit-suisse.com<mailto:vahe.antonian.2 at credit-suisse.com> | www.credit-suisse.com<http://www.credit-suisse.com>

==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================



==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://iqug.org/pipermail/iqug/attachments/20181220/350dadb2/attachment-0001.html>


More information about the IQUG mailing list