Not signed in (Sign In)

Vanilla 1.1.5a is a product of Lussumo. More Information: Documentation, Community Support.

    • CommentAuthorNikolayB
    • CommentTimeMay 23rd 2011
     
    Problems in the way

    Hey, Bjorn!

    Version 1.0.8 is not as stable as 1.0.7. After recording for about 16 days on the charts there are gaps. This can be seen in the screenshots:
    http://img545.imageshack.us/i/92122161.png/
    http://img834.imageshack.us/i/84548366.png/
    http://img843.imageshack.us/i/31631480.png/

    In the log JetProfiler appear on:
    2011-05-20 15:18:29,017 WARN [Logger Daemon] bf - Can't store data fast enough, will have to start rejecting data. Try a bigger polling interval.
    2011-05-20 15:18:29,017 INFO [Logger Daemon] bf - Rejecting data, queued 10529590 bytes
    2011-05-20 15:18:39,002 INFO [Logger Daemon] bf - Rejecting data, queued 10522690 bytes
    2011-05-20 15:18:49,064 INFO [Logger Daemon] bf - Rejecting data, queued 10509590 bytes
    2011-05-20 15:18:59,017 INFO [Logger Daemon] bf - Rejecting data, queued 10502990 bytes
    2011-05-20 15:19:09,017 INFO [Logger Daemon] bf - Rejecting data, queued 10496290 bytes
    2011-05-20 15:19:19,002 INFO [Logger Daemon] bf - Rejecting data, queued 10483290 bytes
    2011-05-20 15:19:29,017 INFO [Logger Daemon] bf - Rejecting data, queued 10470490 bytes
    2011-05-20 15:19:39,018 INFO [Logger Daemon] bf - Rejecting data, queued 10463790 bytes
    2011-05-20 15:19:49,002 INFO [Logger Daemon] bf - Rejecting data, queued 10450590 bytes
    2011-05-20 15:19:54,018 INFO [Logger Daemon] bf - Rejecting data, queued 10450590 bytes
    2011-05-20 15:20:04,018 INFO [Logger Daemon] bf - Rejecting data, queued 10437290 bytes
    2011-05-20 15:20:14,002 INFO [Logger Daemon] bf - Rejecting data, queued 10430790 bytes
    2011-05-20 15:20:19,018 INFO [Logger Daemon] bf - Rejecting data, queued 10424490 bytes
    2011-05-20 15:20:29,002 INFO [Logger Daemon] bf - Rejecting data, queued 10411290 bytes
    2011-05-20 15:20:34,018 INFO [Logger Daemon] bf - Rejecting data, queued 10411290 bytes
    2011-05-20 15:20:44,002 INFO [Logger Daemon] bf - Rejecting data, queued 10398290 bytes
    2011-05-20 15:20:49,018 INFO [Logger Daemon] bf - Rejecting data, queued 10391890 bytes
    2011-05-20 15:20:59,018 INFO [Logger Daemon] bf - Rejecting data, queued 10385290 bytes
    2011-05-20 15:21:09,018 INFO [Logger Daemon] bf - Rejecting data, queued 10372290 bytes
    2011-05-20 15:21:19,018 INFO [Logger Daemon] bf - Rejecting data, queued 10365790 bytes
    2011-05-20 15:21:29,018 INFO [Logger Daemon] bf - Rejecting data, queued 10352990 bytes
    2011-05-20 15:21:39,018 INFO [Logger Daemon] bf - Rejecting data, queued 10340090 bytes
    2011-05-20 15:21:49,018 INFO [Logger Daemon] bf - Rejecting data, queued 10333790 bytes
    2011-05-20 15:21:59,003 INFO [Logger Daemon] bf - Rejecting data, queued 10320390 bytes
    2011-05-20 15:22:04,018 INFO [Logger Daemon] bf - Rejecting data, queued 10320390 bytes
    2011-05-20 15:22:11,034 WARN [Logger Daemon] bk - Running behind, skipping polling, Is the db slow? Try longer polling interval.
    2011-05-20 15:22:14,003 INFO [Logger Daemon] bf - Rejecting data, queued 10307490 bytes
    2011-05-20 15:22:19,019 INFO [Logger Daemon] bf - Rejecting data, queued 10300790 bytes
    2011-05-20 15:22:29,019 INFO [Logger Daemon] bf - Rejecting data, queued 10294090 bytes
    2011-05-20 15:22:39,003 INFO [Logger Daemon] bf - Rejecting data, queued 10281590 bytes
    2011-05-20 15:22:49,019 INFO [Logger Daemon] bf - Rejecting data, queued 10275290 bytes
    2011-05-20 15:22:59,019 INFO [Logger Daemon] bf - Rejecting data, queued 10262490 bytes
    2011-05-20 15:23:09,019 INFO [Logger Daemon] bf - Rejecting data, queued 10250090 bytes
    2011-05-20 15:23:19,019 INFO [Logger Daemon] bf - Rejecting data, queued 10243590 bytes
    2011-05-20 15:23:29,003 INFO [Logger Daemon] bf - Rejecting data, queued 10230690 bytes
    2011-05-20 15:23:34,019 INFO [Logger Daemon] bf - Rejecting data, queued 10224390 bytes
    2011-05-20 15:23:44,019 INFO [Logger Daemon] bf - Rejecting data, queued 10218090 bytes
    2011-05-20 15:23:54,019 INFO [Logger Daemon] bf - Rejecting data, queued 10205090 bytes

    Apparently, there comes a limitation of the inner architecture of the database, because on two different computers is repeated with great precision over a period of data accumulation.

    Screenshots taken from the computer Win2003 SP2, Core i5 2.67Ghz, 4Gb Ram
  1.  
    Hello Nikolay,

    thank you for the detailed report and screenshots! It looks like after 16 days of running, the internal database is having problems storing the recorded information at the same pace as the data is being collected.

    As the logs say, once the waiting data volume reaches 10 megs, it will start rejecting information until it catches up again.

    A workaround is of course to start a new recording once a week.

    I will investigate the causes.
    • CommentAuthorNikolayB
    • CommentTimeJun 1st 2011
     
    Hello Bjorn,

    In my memory were successful measure up to 17 days without any errors.
    Here is a list of screenshots and their results. For the first time the bug had appeared in December 2010.

    7 days 20100405-12 all MyISAM cache.png http://img805.imageshack.us/img805/3308/2010040512allmyisamcach.png
    9 days 20100426-05 all MyISAM cache.png http://img560.imageshack.us/img560/3129/2010042605allmyisamcach.png
    7 days 20100519-26 all MyISAM cache.png http://img863.imageshack.us/img863/8272/2010051926allmyisamcach.png
    12 days 20100616-28 all MyISAM cache.png http://img560.imageshack.us/img560/5350/2010061628allmyisamcach.png
    12 days 20100706-19 all MyISAM cache.png http://img135.imageshack.us/img135/4374/2010070619allmyisamcach.png
    14 days 20100719-02 all MyISAM cache.png http://img135.imageshack.us/img135/2461/2010071902allmyisamcach.png
    15 days 20100816-01 all MyISAM cache.png http://img534.imageshack.us/img534/1124/2010081601allmyisamcach.png
    17 days 20100901-19 all MyISAM cache.png http://img691.imageshack.us/img691/7205/2010090119allmyisamcach.png
    13 days 20100920-04 all MyISAM cache.png http://img851.imageshack.us/img851/4451/2010092004allmyisamcach.png
    13 days 20101004-18 all MyISAM cache.png http://img847.imageshack.us/img847/4052/2010100418allmyisamcach.png
    13 days 20101018-01 all MyISAM cache.png http://img38.imageshack.us/img38/4976/2010101801allmyisamcach.png
    15 days 20101101-17 all MyISAM cache.png http://img26.imageshack.us/img26/5838/2010110117allmyisamcach.png
    20 days, error at 18.5 days 20101125-16 all MyISAM cache.png http://img542.imageshack.us/img542/7601/2010112516allmyisamcach.png
    25 days, error at 18 days 20101217-11 all MyISAM cache.png http://img13.imageshack.us/img13/7486/2010121711allmyisamcach.png
    21 days, error at 16 days 20110112-02 all MyISAM cache.png http://img195.imageshack.us/img195/4552/2011011202allmyisamcach.png
    19 days, error at 18 days 20110202-21 all MyISAM cache.png http://img848.imageshack.us/img848/8725/2011020221allmyisamcach.png
    17.5 days, error at 16 days 20110221-11 all MyISAM cache.png http://img4.imageshack.us/img4/4287/2011022111allmyisamcach.png

    Version 1.0.8 I installed somewhere in March 2010. A bug appeared later. It seems that the theory of version 1.0.8 has not been confirmed.
  2.  
    Hello Nikolay,

    thanks for the detailed information. It seems strange that a problem like that can "just" arise out of nowhere. If it is not tied to any particular release we made, then I have a few guesses:

    * There might be a bottleneck in our internal database that gets hit as your load increases. For example, if your database is receiving more traffic as of December 2010.

    * The machine you are using has gotten a new OS install or other program updates which changes the performance of the machine.
    • CommentAuthorNikolayB
    • CommentTimeJun 3rd 2011 edited
     
    Hello Bjorn!

    Load varies from season to season. Since December 2010 has added several modules to the site, which could lead to increased load on the database. But in general graphs this fact may not be noticed.

    Monitoring is parallel with the two machines. Windows 2008 Server and Windows XP. The results of measurements - the same ...
  3.  
    Hi,

    okay. Interesting that you are getting the same monitoring results on two machines. That might indicate that the problem is structured, reproducible and not random. I guess this supports the theory that after a certain point, the data collected hits a bottleneck in the internal database.

    Thanks!