Listing files from AWS S3 in python using boto3

Boto3

To install Boto3

Install AWS in python

Creating a bucket in S3

aws s3 mb s3://mybucketoutput- 
make_bucket: my-bucket

Listing all the buckets in s3

import boto3s3 = boto3.resource('s3')for bucket in s3.buckets.all():
print(bucket.name)
output-
my-bucket
my-bucket1
my-bucket2

Listing all Keys(In common language folder and files)

import boto3s3 = boto3.client('s3')
bucket = 'my-bucket'
prefix = 'dir1/sub-dir1/'
for obj in s3.list_objects_v2(Bucket=bucket, Prefix=prefix)['Contents']:
print(obj['Key'])
output-
dir1/sub-dir1/s3-file.txt

Reading a file from S3

bucket = 'my-bucket'
prefix = 'dir1/sub-dir1/s3-file.txt'
s3 = boto3.resource('s3')
obj = s3.Object(bucket, prefix)
body = obj.get()['Body'].read()
print(body)
Output-
b'This is a sample text file.\n'

Downloading a file from S3

s3 = boto3.client('s3')
bucket = 'my-bucket'
s3_file = 'dir1/sub-dir1/s3-file.txt'
to_be_downloaded_file = 'to_be_downloaded_file.txt'
s3.download_file(bucket, s3_file, to_be_downloaded_file)
  • Low-level AWS service access
  • Generated from AWS service description
  • Exposes botocore client to the developer
  • Typically maps 1:1 with the AWS service API
  • All AWS service operations are supported by clients
  • Snake-cased method names (e.g. ListBuckets API => list_buckets method)

Resource

  • Higher-level, object-oriented API
  • Generated from resource description
  • Uses identifiers and attributes
  • Has actions (operations on resources)
  • Exposes subresources and collections of AWS resources
  • Does not provide 100% API coverage of AWS services

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store