How to mount s3 bucket in linux EC2 instance
Learn how to mount s3 bucket in linux EC2 instance.
Tested in : EC2 and EC2-beanstalk instances
If you have not created a s3 bucket kindly go through with EC2 documentation.The documentation is very clear and easy to understand .This is the link how to create S3bucket
After creating a new bucket follow the given below steps.
Step 1: Download the latest s3fs package from S3fs link
http://s3fs.googlecode.com/files/s3fs-1.63.tar.gz
Step 2: Untar the package in EC2 instance.
tar -xvzf s3fs-1.63.tar.gz
Step 3: If it is new centos or ubuntu instance. Update the system .
For CentOS or Red Hat yum update all
For Ubuntu sudo apt-get update
Step 4: Now install the dependencies.
In CentOS or Red Hat yum install gcc libstdc++-devel gcc-c++ fuse fuse-devel curl-devel libxml2-devel openssl-devel mailcap
In Ubuntu or Debian apt-get install build-essential gcc libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support build-essential libcurl4-openssl-dev
Step 5: Now change to extracted directory
cd s3fs-1.63
Step 6: Now compiling
./configure --prefix=/usr make make install
Step 7: To check where s3fs command is placed in O.S. It will also tell you the installation is ok.
which s3fs
Step 8: Now getting the access key and secret key.
Click on AWS Menu -> Your AWS Account Name -> Security Credentials
Note: The option is at top Right corner of the aws console page
Here you can see access key and secret key (secret key is visible when you click on show tab)
Copy these both keys seperately
Step 9 : Create a new file in /etc with the name passwd-s3fs
Paste the access key and secret key in this manner .
accesskey:secretkey
Note: colon (:) is require in between of both keys.
for eg.
Access key: bwbshjklwmsysiene7hh
Secret key: jdjbeeosakw3389
vi /etc/passwd-s3fs bwbshjklwmsysiene7hh:jdjbeeosakw3389 save and exit from file. Note: I just gave an eg. of keys by randomely typing the letters. But you have to only paste the keys which you have obtained from "Security Credential" tab -see step 8
Step 10: change the permission of file
chmod 640 /etc/passwd-s3fs
Step 11 : now create a directory to mount S3bucket
mkdir /mys3bucket
Step 12: Now mount the s3 bucket
s3fs bucketname /mys3bucket
Note: you can get bucket name from S3 management console.
Step 13: To check mounted s3 bucket
df -Th /mys3bucket
Step 14 : If you already had some data in s3bucket and it is not visible then you have to set permission in ACL at S3 AWS management console for related s3 bucket.
Can I use this AWS S3 bucket post mount like a normal filesystem (NFS)?
For example : can I copy files from /home/root to this mount point which is the S3 storage/bucket
Hi Sourav,
1. Yes! you can mount the s3 bucket in system.
2. I hope it is /root not /home/root
3. Please install latest package. This is an old post I have still not updated. But I can help you here –
– https://github.com/s3fs-fuse/s3fs-fuse The project is moved to Github now and installation instruction is available in given github link
Regards
Sharad
Connection established, waiting for welcome message…
Response: 220 (vsFTPd 2.2.2)
Command: USER backup
Response: 530 Permission denied.
Error: Could not connect to server
Getting error after connecting to ftp. User default directory set as s3 bucket mounted folder.
Hi, I get the error message “s3fs: MOUNTPOINT: /mys3bucket permission denied.”
I’ll appreciate for any advice.
Hello Slawek,
Check the IAM policy of user which Access key you are using or s3 bucket permission policy in AWS. Do the appropriate changes.
Note: The post is outdated, I will update this post with new s3fs package soon.
Regards
Sharad
My ec2 instance has s3 full access role assigned. However it looks like ec2-user doesn’t have correct rights to the folder /mys3bucket
Thanks for this useful information.
I have created a folder inside s3 bucket, but I can not see these folders in the mounted bucket.
Folders are public.
Do I need to set any permission to make this accessible?
Yes you have to set permission for S3 bucket from your amazon S3 panel also.
Cool, thank you.
Most Welcome
Regards
Sharad
Hi, Thanks for the step by step guide. Will it stay mounted after re-boot?
Thanks Gordon,
Yes you can mount the s3bucket after reboot,you have to write script and put it into /etc/rc.local file.