Today,while working in cloud server.I faced an issue.I was copying some files to s3bucket,which I have mounted by using s3fs package. One every copy command(i.e
cp),I was getting error – fatal error: error writing to /tmp/ccwAjc9Z.s: No space left on device.
The problem was not new for me.I have seen this problem earlier also.The problem occur when any service/program uses the /tmp file.And the /tmp is not writable.
Reason for this fatal error
(1) The Disk space of /tmp partition might be full
(2) /tmp got a new mounted partition, which was not actually earlier. Means,/tmp itself get its partition.
Solution : For this problem ,follow the given below troubleshooting steps
Step 1: First run the
df -h command. The command will show,disk space to mounted partition.
In case /tmp is full (100%). Remove the files/directory from /tmp .
rm -fr /tmp/*
After this, check again. If still issue persist go for Step 2 as given below
Step 2: Sometimes the server do very unusual behavior. Because of continuous file writing in /tmp and because of some other partition issue/ memory issue.
The /tmp will mount to spool partition overflow.
See the below given detail from my server
root@server:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/vda 20G 13G 6.5G 66% / udev 494M 4.0K 494M 1% /dev tmpfs 200M 212K 199M 1% /run none 5.0M 0 5.0M 0% /run/lock none 498M 0 498M 0% /run/shm overflow 1.0M 0 1.0M 0% /tmp root@server:~#
Here,I found that /tmp is mounted to spool partition overflow .
In this case,simply unmount the /tmp .
Use given below two command :
umount /tmp mount -a
Now run again
df -h command.To check,all mounted partition are correctly matched . Now the problem should be solved.
In case,even after removing file and directory from /tmp . Your partition is still showing full disk space.
lsof command and get the DELETED item PID no. and kill it.
lsof | grep deleted|grep tmp|while read abc;do kill -9 $abc;done