what should be the System Usage ?

[lnxcode]

I think the main issue here is that you have one of your scripts that is leaking memory, that is what is more then likely causing the issue. You should remove all the scripts and check to ensure its not going up when there is no site loaded.

If your unable to do that, I would suggest removing all the plugins etc...

The issue can't be on the VPS, as we use a clone to provsion VPS's so if this was an issue there would be 100's of customers complaining about this very issue as well.

Sounds like you have either:

1. A run away process
2. A memory leak
3. Something else running.

I'm not sure what it would be. However, its got to be one of the above. You'll have to find it and address it. Alternatively, you can apply for a re-image and we will be glad to do this for you. We can wipe your whole VPS and clone a new version of it, this can be done for $19.95/one-time.

[mrugesh78]

hello

I checked all scripts. Removed pages and tested by all means. Finally I got the problem.

I identified that my "system usage" under VPS gets 100% in every 6 hours. I identified that the problem is in mysql. If I restart mysql service then it becomes normal.

here it the problem :
process name : mysql

priority : 10
CPU % : 6.5
memory %: 5.3

command :
/usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/lib/mysql/server.readgujarati.com.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock

above is the process details.
Then I run one tunning script to check what is the problem on mysql.
at 98% SYSTEM USEGE, it is giving me following report :

        -- MYSQL PERFORMANCE TUNING PRIMER --
             - By: Matthew Montgomery -

MySQL Version 5.0.51a-community i686

Uptime = 0 days 5 hrs 57 min 3 sec
Avg. qps = 11
Total Questions = 236975
Threads Connected = 1

Warning: Server has not been running for at least 48hrs.
It may not be safe to use these recommendations

To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service

SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 10 sec.
You have 0 out of 237003 that take longer than 10 sec. to complete
Your long_query_time may be too high, I typically set this under 5 sec.

BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html

WORKER THREADS
Current thread_cache_size = 128
Current threads_cached = 7
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine

MAX CONNECTIONS
Current max_connections = 100
Current threads_connected = 1
Historic max_used_connections = 8
The number of used connections is 8% of the configured maximum.
You are using less than 10% of your configured max_connections.
Lowering max_connections could help to avoid an over-allocation of memory
See "MEMORY USAGE" section to make sure you are not over-allocating

MEMORY USAGE
Max Memory Ever Allocated : 75 M
Configured Max Per-thread Buffers : 417 M
Configured Max Global Buffers : 42 M
Configured Max Memory Limit : 459 M
Physical Memory : 512.00 M
Max memory limit seem to be within acceptable norms

KEY BUFFER
Current MyISAM index space = 5 M
Current key_buffer_size = 16 M
Key cache miss rate is 1 : 1057
Key buffer fill ratio = 13.00 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere

QUERY CACHE
Query cache is enabled
Current query_cache_size = 16 M
Current query_cache_used = 9 M
Current query_cache_limit = 1 M
Current Query cache Memory fill ratio = 57.60 %
Current query_cache_min_res_unit = 4 K
MySQL won't cache query results that are larger than query_cache_limit in size

SORT OPERATIONS
Current sort_buffer_size = 1 M
Current read_rnd_buffer_size = 1020 K
Sort buffer seems to be fine

JOINS
Current join_buffer_size = 1.00 M
You have had 18 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.

Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.

OPEN FILES LIMIT
Current open_files_limit = 4096 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_cache value = 1000 tables
You have a total of 134 tables
You have 149 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 64 M
Current tmp_table_size = 64 M
Of 9505 temp tables, 60% were created on disk
Perhaps you should increase your tmp_table_size and/or max_heap_table_size
to reduce the number of disk-based temporary tables
Note! BLOB and TEXT columns are not allow in memory tables.
If you are using these columns raising these values might not impact your
ratio of on disk temp tables.

TABLE SCANS
Current read_buffer_size = 1020 K
Current table scan ratio = 161 : 1
read_buffer_size seems to be fine

TABLE LOCKING
Current Lock Wait ratio = 1 : 4154
You may benefit from selective use of InnoDB.
If you have a high concurrency of inserts on Dynamic row-length tables
consider setting 'concurrent_insert=2'.

Can you tell me what setting I should do to my.cnf file to make less memory usage on Mysql ?

[Shazam]

Originally Posted by mrugesh78

hello

I checked all scripts. Removed pages and tested by all means. Finally I got the problem.

I identified that my "system usage" under VPS gets 100% in every 6 hours. I identified that the problem is in mysql. If I restart mysql service then it becomes normal.

here it the problem :
process name : mysql

priority : 10
CPU % : 6.5
memory %: 5.3

command :
/usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/lib/mysql/server.readgujarati.com.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock

above is the process details.
Then I run one tunning script to check what is the problem on mysql.
at 98% SYSTEM USEGE, it is giving me following report :

        -- MYSQL PERFORMANCE TUNING PRIMER --
             - By: Matthew Montgomery -

MySQL Version 5.0.51a-community i686

Uptime = 0 days 5 hrs 57 min 3 sec
Avg. qps = 11
Total Questions = 236975
Threads Connected = 1

Warning: Server has not been running for at least 48hrs.
It may not be safe to use these recommendations

To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service

SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 10 sec.
You have 0 out of 237003 that take longer than 10 sec. to complete
Your long_query_time may be too high, I typically set this under 5 sec.

BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html

WORKER THREADS
Current thread_cache_size = 128
Current threads_cached = 7
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine

MAX CONNECTIONS
Current max_connections = 100
Current threads_connected = 1
Historic max_used_connections = 8
The number of used connections is 8% of the configured maximum.
You are using less than 10% of your configured max_connections.
Lowering max_connections could help to avoid an over-allocation of memory
See "MEMORY USAGE" section to make sure you are not over-allocating

MEMORY USAGE
Max Memory Ever Allocated : 75 M
Configured Max Per-thread Buffers : 417 M
Configured Max Global Buffers : 42 M
Configured Max Memory Limit : 459 M
Physical Memory : 512.00 M
Max memory limit seem to be within acceptable norms

KEY BUFFER
Current MyISAM index space = 5 M
Current key_buffer_size = 16 M
Key cache miss rate is 1 : 1057
Key buffer fill ratio = 13.00 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere

QUERY CACHE
Query cache is enabled
Current query_cache_size = 16 M
Current query_cache_used = 9 M
Current query_cache_limit = 1 M
Current Query cache Memory fill ratio = 57.60 %
Current query_cache_min_res_unit = 4 K
MySQL won't cache query results that are larger than query_cache_limit in size

SORT OPERATIONS
Current sort_buffer_size = 1 M
Current read_rnd_buffer_size = 1020 K
Sort buffer seems to be fine

JOINS
Current join_buffer_size = 1.00 M
You have had 18 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.

Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.

OPEN FILES LIMIT
Current open_files_limit = 4096 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_cache value = 1000 tables
You have a total of 134 tables
You have 149 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 64 M
Current tmp_table_size = 64 M
Of 9505 temp tables, 60% were created on disk
Perhaps you should increase your tmp_table_size and/or max_heap_table_size
to reduce the number of disk-based temporary tables
Note! BLOB and TEXT columns are not allow in memory tables.
If you are using these columns raising these values might not impact your
ratio of on disk temp tables.

TABLE SCANS
Current read_buffer_size = 1020 K
Current table scan ratio = 161 : 1
read_buffer_size seems to be fine

TABLE LOCKING
Current Lock Wait ratio = 1 : 4154
You may benefit from selective use of InnoDB.
If you have a high concurrency of inserts on Dynamic row-length tables
consider setting 'concurrent_insert=2'.

Can you tell me what setting I should do to my.cnf file to make less memory usage on Mysql ?

This is above my pay grade, but someone else will take care of this. But your memory usage says that the "Max Memory Ever Allocated" is 75MB. That isn't a lot.

How much PHP memory is your VPS set for?

[lnxcode]

If the MySQL is an issue, feel free to update it. What you can do is:

1. Login to your WHM
2. Update Config
3. Change from "Never" to "Inherit"
4. Save
5. Run the update.

That should update you to the latest version that cPanel has.

[Shazam]

Originally Posted by lnxcode

If the MySQL is an issue, feel free to update it. What you can do is:

1. Login to your WHM
2. Update Config
3. Change from "Never" to "Inherit"
4. Save
5. Run the update.

That should update you to the latest version that cPanel has.

That's 5.0.67 community, I believe. This was recently released and it's the best available.

If you want to update Apache and PHP, you'd have to use cPanel's Easy Apache feature, but be careful. It can sometimes go awry, and you may be caught in a bad situation if you don't know how to handle problems.

In the end, you may just want to invest in buying a maintenance contract with us, and let our techs figure this out for you.

[mrugesh78]

From where I can know that how much php memory is in my VPS is set ?

I think its some memory issue and nothing to do with upgrade. isn't it ?

[Shazam]

Originally Posted by mrugesh78

From where I can know that how much php memory is in my VPS is set ?

I think its some memory issue and nothing to do with upgrade. isn't it ?

WHM has a PHP Configuration Editor when you can check that stuff. If you didn't change it, the memory settings should be fairly modest.

[Smallbizuser]

Not sure if this helps, but I found that Yahoo slurp was killing my resources last month!

It was a VERY aggressive crawler... and this happened one day to the next (no change in scripts).

You may want to check your robots.txt file at the same time you're looking at MySQL config (that part is a little over my head).

I added this line:
User-agent: Slurp
Crawl-delay: 10

(I discovered this by monitoring the resources & AWstats at the same time)

[mrugesh78]

Originally Posted by Shazam

WHM has a PHP Configuration Editor when you can check that stuff. If you didn't change it, the memory settings should be fairly modest.

at present my memory_limit inside PHP configuration editor is : 256M

I think before two month when my VPS was going to the black zone, I informed to support department and they have increased it. Bcos they replied me that : "we have increased memory_limit" So I do not know what was the default. May be it was 128M. May be.

But this can affect mysql process ? bcos I think PHP is different then mysql and the proess consuming is of mysql. Can you guide me please ?

[mrugesh78]

Originally Posted by Smallbizuser

Not sure if this helps, but I found that Yahoo slurp was killing my resources last month!

It was a VERY aggressive crawler... and this happened one day to the next (no change in scripts).

You may want to check your robots.txt file at the same time you're looking at MySQL config (that part is a little over my head).

I added this line:
User-agent: Slurp
Crawl-delay: 10

(I discovered this by monitoring the resources & AWstats at the same time)

hello,
I also checked my awstats and it is showing me that :

Googlebot	96815+41(hits)	5.96 GB (bandwidth)	22 Dec 2008 - 10:31
Yahoo Slurp	35368+427(hits)	1.58 GB (bandwidth)

But I think this is bandwidth and not memory. Can It affect to memory as well ? I am sorry, I donot know in detail. I think there is NO robot.txt on my root. What I shoud do ? this consumption is normal ?