开发者

AppEngine: Getting only 1KB when uploading from an fetched URL to S3

开发者 https://www.devze.com 2023-02-15 03:46 出处:网络
I´m trying to upload a file which is fetched from a server. But when the fetched file is bigger than 1MB there will be only a 1KB small file saved at S3. When it´s smaller than 1MB, the file will be

I´m trying to upload a file which is fetched from a server. But when the fetched file is bigger than 1MB there will be only a 1KB small file saved at S3. When it´s smaller than 1MB, the file will be saved properly.

I ´ve searched and tried different ways, but none off them is working. Here is the class which is triggered by the user clicking a button.

` class Fetch_by_button(webapp.RequestHandler):

def get(self):
    #1. fetch the file
    try: 
        result = urlfetch.fetch('http://someurl.com/music.mp3') # file with 2MB
        #result = urlfetch.fetch('http://anotherurl.com/documen开发者_开发知识库t.pdf') #file with 900KB is working
    except DownloadError:
        print("URL fetch doesnt work properly")

    #2. search datastore UserData-model by userobject to get s3 credentials
    usr = users.get_current_user()
    query = UserData.gql("WHERE account =:1", usr)
    qr = query.get()

    #3. establish connection to s3
    conn = S3Connection(qr.s3_accesskey, qr.s3_secret_accesskey)

    #4. create bucket
    ak_lowercase = str.lower(str(qr.s3_accesskey))
    individual_bucket_name = ak_lowercase + '_' + usr.nickname()
    bucket = conn.create_bucket(individual_bucket_name)

    #5. generate key and assign to already created bucket
    k = Key(bucket)

    #6. name the key by some examplary filename
    k.key = 'justsomefilename'

    #7. save the correspondent value to the created key
    k.set_contents_from_string(result.content)`

I just don´t have any clue why this happens. Something with my code, some AppEngine restrictions, anything with boto, ... Any suggestions?

Thanks in advance.


The maximum request size on app engine is 1mb for urlfetch. I'm guessing your S3Connection class uses that to do the actual transfer.

Responses can be up to 32MB, that's why the download works.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号