How to mount s3 bucket in linux EC2 instance

How to mount s3 bucket in linux EC2 instance

Learn how to mount s3 bucket in linux EC2 instance.
Tested in : EC2 and EC2-beanstalk instances

If you have not created a s3 bucket kindly go through with EC2 documentation.The documentation is very clear and easy to understand .This is the link how to create S3bucket

After creating a new bucket follow the given below steps.

Step 1: Download the latest s3fs package from S3fs link

http://s3fs.googlecode.com/files/s3fs-1.63.tar.gz

Step 2: Untar the package in EC2 instance.

tar -xvzf s3fs-1.63.tar.gz

Step 3: If it is new centos or ubuntu instance. Update the system .

For CentOS or Red Hat

yum update all
For Ubuntu

sudo apt-get update

Step 4: Now install the dependencies.

In CentOS or Red Hat

yum install gcc libstdc++-devel gcc-c++ fuse fuse-devel curl-devel libxml2-devel openssl-devel mailcap
In Ubuntu or Debian

apt-get install build-essential gcc libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support build-essential libcurl4-openssl-dev

Step 5: Now change to extracted directory

cd s3fs-1.63

Step 6: Now compiling

./configure  --prefix=/usr

make

make install

Step 7: To check where s3fs command is placed in O.S. It will also tell you the installation is ok.

which s3fs

Step 8: Now getting the access key and secret key.
Click on AWS Menu -> Your AWS Account Name -> Security Credentials

Note:
The option is at top Right corner of the aws console page

Here you can see access key and secret key (secret key is visible when you click on show tab)

Copy these both keys seperately

Step 9 : Create a new file in /etc with the name passwd-s3fs

Paste the access key and secret key in this manner .

accesskey:secretkey

Note: colon (:) is require in between of both keys.

for eg.

Access key: bwbshjklwmsysiene7hh

Secret key
: jdjbeeosakw3389

vi /etc/passwd-s3fs

bwbshjklwmsysiene7hh:jdjbeeosakw3389

save and exit from file.

Note: I just gave an eg. of keys by randomely typing the letters. But you have to only paste the keys which you have obtained from "Security Credential" tab -see step 8

Step 10: change the permission of file

chmod 640 /etc/passwd-s3fs

Step 11 : now create a directory to mount S3bucket

mkdir /mys3bucket


Step 12:
Now mount the s3 bucket

s3fs bucketname /mys3bucket

Note: you can get bucket name from S3 management console.

Step 13: To check mounted s3 bucket

df -Th /mys3bucket

Step 14 : If you already had some data in s3bucket and it is not visible then you have to set permission in ACL at S3 AWS management console for related s3 bucket.

12 thoughts on “How to mount s3 bucket in linux EC2 instance”

  1. Can I use this AWS S3 bucket post mount like a normal filesystem (NFS)?

    For example : can I copy files from /home/root to this mount point which is the S3 storage/bucket

    Reply
  2. Connection established, waiting for welcome message…
    Response: 220 (vsFTPd 2.2.2)
    Command: USER backup
    Response: 530 Permission denied.
    Error: Could not connect to server

    Getting error after connecting to ftp. User default directory set as s3 bucket mounted folder.

    Reply
    • Hello Slawek,

      Check the IAM policy of user which Access key you are using or s3 bucket permission policy in AWS. Do the appropriate changes.
      Note: The post is outdated, I will update this post with new s3fs package soon.

      Regards
      Sharad

      Reply
      • My ec2 instance has s3 full access role assigned. However it looks like ec2-user doesn’t have correct rights to the folder /mys3bucket

        Reply
  3. Thanks for this useful information.

    I have created a folder inside s3 bucket, but I can not see these folders in the mounted bucket.

    Folders are public.

    Do I need to set any permission to make this accessible?

    Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.