Mongodb Hardware recommendations for production server

i) Disable NUMA -- Running MongoDB on a system with Non-Uniform Access Memory (NUMA) can cause a number of operational problems, including slow performance for periods of time and high system process usage. So,it is recommended to turn off NUMA

ii) Allocate swap space -- Allocating swap space can avoid issues with memory contention and can prevent the OOM Killer on Linux systems from killing mongod. The method mongod uses to map memory files to memory ensures that the operating system will never store MongoDB data in swap space.

iii) RAID10 -- RAID-5 and RAID-6 do not typically provide sufficient performance to support a MongoDB deployment.MongoDB deployments should use disks backed by RAID-10.If it is  a AWS EC2 machines you can take the provisioned iops  storage .

iv) Noop scheduler  -- Local block devices attached to virtual machine instances via the hypervisor should use a noop scheduler for best performance. The noop scheduler allows the operating system to defer I/O scheduling to the underlying hypervisor.

v) ulimit  -- Set the file descriptor limit, -n, and the user process limit (ulimit), -u, to 64000.

vi) Disable transparent huge pages -- MongoDB performs better with normal (4096 bytes) virtual memory pages.

vii) Turnoff atime : Operating systems maintain file system metadata that records when each file was last accessed, this is atime. The importance of the noatime setting is that it eliminates the need by the system to make writes to the filesystem for files which are simply being read. Since writes can be somewhat expensive, this can result in measurable performance gains.Turn off atime for the storage volume containing the database files.

viii)read-ahead  Ensure that read-ahead settings for the block devices that store the database files are appropriate. For random access use patterns, set low read ahead values. A read ahead of 32 (16 kB) often works well for MMAP() engine and A read ahead of 0 for wiredtiger storage engine.

ix) File System  Use the XFS file system over the EXT4 for better performance 


Apply  the changes 

1.File system type should be  XFS 

With the WiredTiger storage engine, use of XFS is strongly recommended to avoid performance issues that may occur when using EXT4 with WiredTiger.

2. Disable Transparent Huge Pages.

MongoDB performs better with normal (4096 bytes) virtual memory pages 

Temporary  disable the Transparent Huge Pages by executing the below commands .

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

To disable the Transparent Huge Pages permanently .

cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
cat /sys/kernel/mm/transparent_hugepage/defrag
always madvise [never]

add the transparent_hugepage=never in /etc/grub.conf ans scripts

###########
vi /etc/init.d/disable-transparent-hugepages
#!/bin/sh
### BEGIN INIT INFO
# Provides: disable-transparent-hugepages
# Required-Start: $local_fs
# Required-Stop:
# X-Start-Before: mongod mongodb-mms-automation-agent
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Disable Linux transparent huge pages
# Description: Disable Linux transparent huge pages, to improve
# database performance.
### END INIT INFO

case $1 in start)
if [ -d /sys/kernel/mm/transparent_hugepage ]; then
thp_path=/sys/kernel/mm/transparent_hugepage
elif [ -d /sys/kernel/mm/redhat_transparent_hugepage ]; then
thp_path=/sys/kernel/mm/redhat_transparent_hugepage
else
return 0
fi

echo 'never' > ${thp_path}/enabled
echo 'never' > ${thp_path}/defrag

unset thp_path
;; 
esac

chmod 755 /etc/init.d/disable-transparent-hugepages 
chkconfig --add disable-transparent-hugepages

###########

3.Disable NUMA in your BIOS 

add the below numa=off in /etc/grub.conf 
to check the numa enable or not by using the cat /var/log/dmesg | grep -i numa.

4. DIsable the SELINUX 

Disable SELinux by setting the SELINUX setting to disabled in /etc/selinux/config.

SELINUX=disabled

5.Turnoff atime

add the noatime in /etc/fstab for the data partition.

to verify the noatime disabled or not execute mount command.

4. tcp_keepalive_time 

to check the keep alive time execute cat /proc/sys/net/ipv4/tcp_keepalive_time

adjust the tcp_keepalive_time temporarily 

 tcp_keepalive_time echo 900> /proc/sys/net/ipv4/tcp_keepalive_time

permanently  add the net.ipv4.tcp_keepalive_time=300 in /etc/sysctl.conf

5.Noop scheduler

check the no scheduler value cat /sys/block/xvdg/queue/scheduler

should be noop for VM and deadline for physical server

6.Setting of the readahead setting for random I/O 'S

to verify the readhead execute the command:  blockdev --report

For the WiredTiger storage engine:

Read-Ahead

Set the readahead setting to 0 . Setting a higher readahead benefits sequential I/O operations. However, since MongoDB disk access patterns are generally random, setting a higher readahead provides limited benefit. As such, for most workloads, a readahead of 0  provides optimal MongoDB performance.

For the MMAPv1 storage engine:

Ensure that readahead settings for the block devices that store the database files are appropriate. For random access use patterns, set low readahead values. A readahead of 32 (16 kB) often works well.
For a standard block device, you can run sudo blockdev --report to get the readahead settings and sudo blockdev --setra <value> <device> to change the readahead settings. Refer to your specific operating system manual for more information.

adjust the read add count blockdev --setra 32 /dev/xvdf need add in /etc/rc.local