DataRenderService Times API is very slow and pegs CPU


#1

PhantomBot Version: 2.4.2 (Revision: 5c75a7b2) – I also checked that the bug still appears in the latest nightly.
OS Version: Linux 4.19.0-1-amd64
Java Version: 1.8.0_181-8u181-b13-2~deb9u1-b13
Browser and Version (for Panel Support): n/a
Stock PhantomBot: No, but nearly – only added some scripts under commands/ and they shouldn’t be related.

Let us know what help you need:

Hey there – when the DataRenderService runs, the Times API stage seems to be taking forever and maxing out my machine’s CPU – here’s a CPU graph: https://i.imgur.com/237ZD17.png (Nothing else is running on that machine.)

Here’s an excerpt from the event log, note the timestamps:

[02-06-2019 @ 09:42:27.978 GMT] [dataServiceHandler.js:85] DataRenderService: Commands API status : success
[02-06-2019 @ 09:42:28.060 GMT] [dataServiceHandler.js:100] DataRenderService: Quotes API status : success
[02-06-2019 @ 09:42:33.758 GMT] [dataServiceHandler.js:113] DataRenderService: Points API status : success
[02-06-2019 @ 10:20:43.630 GMT] [dataServiceHandler.js:134] DataRenderService: Times API status : success
[02-06-2019 @ 10:20:49.985 GMT] [dataServiceHandler.js:137] DataRenderService: Ranks API status : success
[02-06-2019 @ 10:20:49.985 GMT] [dataServiceHandler.js:139] DataRenderService: Handler Process Complete

So it takes about 40 minutes for the Times API to run – and the beginning and end times all line up with the periods of 100% CPU on the graph. (The run at about 06:00 took 2½ hours, also entirely in the Times API – that was the first time that DRS fired after I upgraded to the nightly, which might or might not be coincidence.)

Any idea what’s going on here? I’m happy to send you more debugging information – let me know what you need. Thank you!


#2

It has to query all of the tables listed there and convert all of the data to JSON which is a CPU and memory intensive operation. I expect the CPU usage to be spike. Since a large chunk of memory is also allocated and freed, the garbage collector more than likely is kicked in, adding to CPU usage. Times also has to walk your Ranks data and report on ranks - so it takes longer to extrapolate and calculate that data and create a second JSON table to be sent.


#3

That’s also 100% of one core, not your whole CPU.


#4

Thanks for the explanation! I found a way to speed things up substantially and finish in <1% of the time – filed as PR #2112 so we can continue the discussion there.