Friday, August 14, 2009

Reading other processes memory

As you probably have experienced yet MySQL does not always provide all internal information as you might want to have them and as you are used to have from other RDBMS.

MySQL plans to improve this implementing the/a performance schema and its probably already partly done in MySQL 5.4. But who knows when this will be finished and what it contains at all...

What is not provided to me I want to gather myself... But how? Other RDBMS provide interfaces to attach applications directly to their memory to retreive information. But MySQL does not. So I was looking for a way to read an other process memory.

I have no clue about programming and thus changing MySQL code was out of focus. Further I am looking for a solution you can use immediately on a running systems at consulting gigs. Some tries to read /proc/<pid>/mem with a little php script failed.

An article by Domas M. helped me. I do not have to write something myself I can use a tool already exsting to do the work. But gdb is not installed on every machine and usually not at all on production machines. Further gdb is probably an overkill to just read memory of other processes.

But an other application to do this job I did not find. I just found some comments that ptrace is the way to do it. Ptrace (man ptrace) is not a program (as for example strace) but an operating system function call.

When you are interested how I found out how to do it please continue reading here.

4 comments:

Morgan said...

Very nice! I can see the real use with some of the InnoDB stats. The random sampling interval makes it very difficult to get good numbers to use for cacti/RRDTool graphs.

Shinguz said...

Hi Morgan,

Do not get this. With the script read_process_memory.sh you can sample at a specific interval with a timestamp in the record and write a *.csv. That must be ideal for Cacti or any other monitoring/graphing tool, is it not? You could even extend it to write directly to a database...

Please elaborate a bit more your concern.

Morgan said...

I just read what I wrote, sorry for not making sense. What I meant:

The *default* INNODB STATUS method of showing statistics is for anywhere between 0 seconds to 60 seconds makes it difficult. 0-10 seconds is usually useless because it averages too much, and it's entirely unpredictable what interval InnoDB uses.

Your script allows me to just get the current value (not an average). Brilliant!

Shinguz said...

A colleague pointed out that I was missing some information:

The operative worklog task is WL#2360 and its dependencies, WL#4034 is a raw-idea bin item of no current significance.

The complete specification of the performance schema can be found here:

WL#2333: SHOW ENGINE ... LOCK STATUS
WL#2360: Performance Schema
WL#2515: Performance statements
WL#3249: SHOW PROCESSLIST should show memory
WL#4674: PERFORMANCE_SCHEMA Setup For Actors
WL#4678: PERFORMANCE_SCHEMA Instrumenting File IO
WL#4813: PERFORMANCE_SCHEMA Instrumenting Stages
WL#4816: PERFORMANCE_SCHEMA Summaries
WL#4895: PERFORMANCE_SCHEMA Instrumenting Table IO
WL#4878: PERFORMANCE_SCHEMA Trace
WL#4896: PERFORMANCE_SCHEMA Instrumenting Net IO

There is an implementation of WL#2360 which was demonstrated 11 months ago at the MySQL developer conference:

Blog post 1
Blog post 2
Blog post 3
Blog post 4
Blog post 5
Blog post 6
Blog post 7

The source code of the server including the Performance Schema is available for download from lauchpad.