[IQUG] Memory sizing in IQ

Rey reywang at optonline.net
Wed May 1 14:37:06 MST 2013


written to both IQ temp store and IQ temp cacheat the same time

In what condition, we should have the I/O at temp store here.
I suspect regardless we have very big temp cache (over 100GB), we can't 
avoid i/o at temp store; even we do not exceed 50GB usages at temp cache.


On 5/1/2013 11:16 AM, Louie, David wrote:
>
> As Mark indicated IQ will use cache first for loads and the after it 
> exhausts temp cache will start using IQ temp store.  Only for temp 
> table related activity will it write to both at the same time.  For us 
> we were heavily allocated so were able to deviate from the 40/60 rule 
> and allocate less to temp.  We didn't make temp cache the same as temp 
> IQ store for the very reason that we need the extra cache for 
> situations where only temp cache is used.
>
> *From:*Rey [mailto:reywang at optonline.net]
> *Sent:* Wednesday, May 01, 2013 11:06 AM
> *To:* Mumy, Mark
> *Cc:* iqug at iqug.org; Louie, David
> *Subject:* Re: [IQUG] Memory sizing in IQ
>
> Check to see if your IQ temp cache is close to your IQ temp store 
> size.    We had our IQ temp cache over allocated at 450 GB when our IQ 
> temp store was only 100 GB.  We were advised by Mark Mumy that IQ temp 
> cache iswritten to both IQ temp store and IQ temp cacheat the same 
> time( he indicates this behavior may change in 16).
>
> Is it that is from your suggestion we should keep equal size of temp 
> cache and temp store?
> Any bad effect if temp store is way too high (such as 4-5 times)?
> There is never an issue for ASE. I am afraid IQ is behaved badly in 
> such condition; and no white paper addressed this.
>
> On 5/1/2013 10:25 AM, Mumy, Mark wrote:
>
>     Rey,
>
>     What is it that you are asking to "fix"?  I am not sure I
>     understand what the question is.
>
>     Mark
>
>     **
>
>     *Mark Mumy, *Technical Director, Enterprise Systems Group, *SAP**
>     *M +1 347-820-2136 | E mark.mumy at sap.com <mailto:mark.mumy at sap.com>
>
>     https://sap.na.pgiconnect.com/I825063
>
>     Conference tel: 18663127353,,7090196396#
>
>
>     Description: Description: Description: Description: Description:
>     sap_09_logo.JPG
>
>     *From:*iqug-bounces at iqug.org <mailto:iqug-bounces at iqug.org>
>     [mailto:iqug-bounces at iqug.org] *On Behalf Of *Rey
>     *Sent:* Wednesday, May 01, 2013 9:20 AM
>     *To:* iqug at iqug.org <mailto:iqug at iqug.org>; Louie, David
>     *Subject:* Re: [IQUG] Memory sizing in IQ
>
>     This is bad, I saw something weird at my system. Our temp store is
>     2-3 times bigger than temp cache.
>     Is it hardware sizing guide should mention this with more test
>     benchmarks?
>
>     I doubt we will upgrade to IQ 16 shortly, can SAP fix this at IQ
>     154 level?
>
>     Rey
>     On 4/30/2013 12:41 PM, Louie, David wrote:
>
>         Check to see if your IQ temp cache is close to your IQ temp
>         store size.    We had our IQ temp cache over allocated at 450
>         GB when our IQ temp store was only 100 GB.  We were advised by
>         Mark Mumy that IQ temp cache is written to both IQ temp store
>         and IQ temp cache at the same time ( he indicates this
>         behavior may change in 16).  We've reduced our IQ temp cache
>         to 150 GB which is closer to our 100 GB of IQ temp store and
>         allocated that RAM back to main cache.  As expected we have
>         seen a tremendous reduction in IO since increasing the main cache.
>
>         David
>
>         *From:*iqug-bounces at iqug.org <mailto:iqug-bounces at iqug.org>
>         [mailto:iqug-bounces at iqug.org] *On Behalf Of *Macalalad, Jun
>         *Sent:* Tuesday, April 30, 2013 9:51 AM
>         *To:* Ron Watkins; 'Toni Berry'; 'IQ Users Group'
>         *Subject:* Re: [IQUG] Memory sizing in IQ
>
>         Does this mean that IQ needs to be by itself in a host server
>         -- without Informatica/Business Objects with SQL Server
>         repository? Else it will starve others of resources hence
>         require restart (of IQ or the host) more often?  Sorry if I am
>         painting a grim picture --- resource-wise ...
>
>         Jun
>
>         *From:*iqug-bounces at iqug.org <mailto:iqug-bounces at iqug.org>
>         [mailto:iqug-bounces at iqug.org] *On Behalf Of *Ron Watkins
>         *Sent:* Tuesday, 30 April 2013 9:17 PM
>         *To:* 'Toni Berry'; 'IQ Users Group'
>         *Subject:* Re: [IQUG] Memory sizing in IQ
>
>         There is never any problem with over allocating cache. I
>         believe the sizing guide is based on an cost-benefit model.
>
>         You should see some performance benefit, although I cannot say
>         how much that will be exactly.
>
>         In theory, you could allocate as much cache as the size of
>         your data, in which case once the data was initially pulled
>         into cache it would stay there and IQ would effectively have
>         its entire store in cache.
>
>         I believe the sizing guide is based on an cost-benefit model.
>
>         As for the utilization, main cache will start at 0% and grow
>         as data is pulled in from disk. Since data is never removed
>         unless the space is needed for another object, it will grow to
>         100% and stay there. That is normal. The rule for main cache
>         is to remove data only when the pages are needed by another
>         object. I believe it uses an algorithm called
>         Least-Recently-Used, so when IQ replaces pages in cache with
>         other objects, it will choose the least recently used object
>         to remove. There is no reason to forcibly remove data from
>         cache in normal operations, so that is why it will stay at
>         100% untill a restart occurs.
>
>         Temp cache is different in that an object there only exists as
>         long as the connection is making use of that object. When the
>         connection goes away, the space associated with that
>         connection is dropped, thus freeing up TEMP cache. This is why
>         you should see TEMP cache fluctuate whereas MAIN cache stays
>         at 100%.
>
>         Ron
>
>         *From:*iqug-bounces at iqug.org <mailto:iqug-bounces at iqug.org>
>         [mailto:iqug-bounces at iqug.org] *On Behalf Of *Toni Berry
>         *Sent:* Tuesday, April 30, 2013 1:03 AM
>         *To:* IQ Users Group
>         *Subject:* [IQUG] Memory sizing in IQ
>
>         Hi All,
>
>         In the Sybase IQ 15 sizing Guide white paper, the general
>         memory guideline is 4-8GB per core.  As we have a 4-core
>         configuration we allocate 22GB to Sybase IQ, so this
>         configuration meets the general guideline of sizing between
>         16-32GB.
>
>         The server that host our Sybase IQ instance will gain more
>         RAM, so I'm looking at allocating Sybase IQ around 32GB RAM.
>
>         I have the opportunity to raise the allocation to 48GB for
>         Sybase IQ  (Say a split of IQ main cache 30GB, IQ Temp Cache
>         18GB).
>
>         My question is there any significant performance gains
>         allocating more memory than what is in the general guideline?
>
>         With our 22GB allocation, IQ Main cache (12GB) is always 100%
>         utilised, whereas IQ Temporary Cache (10GB) varies depending
>         on activity though rarely at 100%
>
>         Regards,   Toni.
>
>         --
>         *Toni Berry*
>         *Apps Support Team Leader - Data Warehouse*
>
>         *E:*toni.berry at aapt.com.au
>         <https://mail.google.com/mail/?view=cm&fs=1&tf=1&to=toni.berry@aapt.com.au>
>
>         *P:* +61 2 9009 1248
>
>         *M:* +61 410 047 511
>
>         30 Ross St, Glebe, NSW 2037, Australia <http://g.co/maps/4s8k3>
>
>         This communication, including any attachments, is confidential. If you are not the intended
>
>         recipient, you should not read it - please contact me immediately, destroy it, and do not
>
>         copy or use any part of this communication or disclose anything about it.
>
>           
>
>         THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL,
>         PROPRIETARY, AND MAY BE PRIVILEGED. If this message was
>         misdirected, BlackRock, Inc. and its subsidiaries,
>         ("BlackRock") does not waive any confidentiality or privilege.
>         If you are not the intended recipient, please notify us
>         immediately and destroy the message without disclosing its
>         contents to anyone. Any distribution, use or copying of this
>         e-mail or the information it contains by other than an
>         intended recipient is unauthorized. The views and opinions
>         expressed in this e-mail message are the author's own and may
>         not reflect the views and opinions of BlackRock, unless the
>         author is authorized by BlackRock to express such views or
>         opinions on its behalf. All email sent to or from this address
>         is subject to electronic storage and review by BlackRock.
>         Although BlackRock operates anti-virus programs, it does not
>         accept responsibility for any damage whatsoever caused by
>         viruses being passed.
>
>
>
>
>         _______________________________________________
>
>         IQUG mailing list
>
>         IQUG at iqug.org  <mailto:IQUG at iqug.org>
>
>         http://iqug.org/mailman/listinfo/iqug
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://iqug.org/pipermail/iqug/attachments/20130501/afb0ce4f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2409 bytes
Desc: not available
URL: <http://iqug.org/pipermail/iqug/attachments/20130501/afb0ce4f/attachment-0001.jpe>


More information about the IQUG mailing list