How to upload External Image URL Directly into S3 Bucket using python

Today we will try to understand how we can Upload Any External Image URL Directly into S3 Bucket using python

See below Flow diagram for the same.

For implementing this functionality we required below Python Components / Libraries.

1. pymysql
2. boto3
3. urllib3
4. urllib.parse import urlparse
5. requests

Note : Please indent python code properly in your editor.

Follow below steps to implement this functionality in your application

Step 1:

Define your AWS Credentials in Constants

AWS_ACCESS_KEY_ID = ‘XXXXXXXXXX’
AWS_SECRET_ACCESS_KEY = ‘XXXXXXXXXX’
AWS_BUCKET_NAME = ‘XXXXXXXXXX’
AWS_REGION_CODE = ‘XXXXXXXXXX’
AWS_SIGNATURE_VERSION = ‘XXXXXXXXXX’

Step 2:

Create Main python function

readUploadFileOnS3Bucket(‘http://www.abc.com/abc.jpg’)

Step 3:

Now we will create new function for validating Image URL In my case i have created “validateURL(url)” function which will validate is image URL valid or not.

def validateURL(url):
isImgValid = 0
http = urllib3.PoolManager()
r = http.request(‘GET’, url)
if(r.status == 200):
isImgValid = 1
return isImgValid

Step 4:

Through below function we will read the image and upload it on S3 Bucket . my function name is “readUploadFileOnS3Bucket(file_url):”

def readUploadFileOnS3Bucket(file_url):
parseUrl = urlparse(file_url);
tempS3Key = parseUrl.path
isImagePresent = validateURL(internet_image_url)

if(isImagePresent == 1):
req_for_image = requests.get(internet_image_url, stream=True)
file_object_from_req = req_for_image.raw
req_data = file_object_from_req.read()

try:
tempS3Key = tempS3Key[1:]
s3Bucket = s3.Bucket(bucket_name_to_upload_image_to) // Name of the S3 Bucket
response = s3Bucket.put_object(Key=tempS3Key, Body=req_data,ACL=’public-read’)
return response
except ClientError as e:
response = 0
return response

In above Function inside Response which i have marked in bold you will get S3 Bucket Object Data. That you can store it in Database for referring image path on main web site.

If any queries contact me on

shardulk10@gmail.com