Best way to handle many ASIHttpRequests and core data operations in ipad app

The current app I'm developing for the iPad involves handling many network requests and persisting the processed results in core data.

The scenario is follows - the application needs to download images for objects I'm displaying in a grid view, which can show a total of 30 obje开发者_运维问答cts. Each object can consist of up to 15 png images (also in a grid). Due to the way the server is implemented (meaning I didn't implement it and can't change it easily), each image must be requested separately, so I need to make up to 15 requests per object versus just 1 request to download all 15 images.

For each object, I'm currently using an ASINetworkQueue to queue up the 15 image requests. Once the queue finishes, I create a thumbnail snapshot of the object with its images to display in the grid, then persist all the png files to core data.

I'm currently running everything on the main thread except the network requests which are handled by ASI asynchronously, but since there are so many requests, the app UI is essentially locked until all the requests are processed and results saved to core data.

One solution I came across was doing the core data operations and writes in a separate thread or using grand central dispatch. Another is to only download the images for the visible objects, and download the rest when the user scrolls down.

I'm looking for other suggestions to help keep the main ui responsive, or better ways to structure the network and core data operations. Thanks.

---

First of all, avoid storing large blobs in Core Data, saving the thumbnails is fine (although you should optimize your model for it), but you should store the full image once it's reconstructed in the Documents folder.

You should definitely use a queue, either NSOperationQueue or ASI network queue. I do something similar in my app which has multiple dependencies. So, for each of the 30 objects you need to have a block (or work function) get called when the 15 images have downloaded. You ideally want to do this work off the main thread. Put all these requirements together, and I'd say you need at least two queues, one for network requests and one for worker blocks, and you should use NSBlockOperations which makes the whole thing much easier. So, the code would be something like this...

// Loop through the objects
for (NSArray *objectParts in objectsToDownload) {

    // Create our Object
    Object *obj = [Object insertIntoManagedObjectContext:self.moc];

    // This is the block which will do the post processing for the object
    NSBlockOperation *processBlock = [NSBlockOperation blockOperationWithBlock:^{

        // Do post processing here, be very careful with multi-threading CoreData
        // it's likely you'll need some class to dispatch you MOCs which have all
        // all the observers set up.

        // We're gonna assume that all the sub-images have been stored in a instance
        // variable:
        [obj performPostProcessing];

    }];

    // Given the list of 15 images which form each object
    for (NSURL *part in objectParts) {

        // Create the ASI request for this part
        ASIHTTPRequest *request = [ASIHTTPRequest requestWithURL:part];

        // Configure the request
        [request setDelegate:self];
        [request setDidFailSelector:@selector(partRequestDidFail:)];
        [request setDidFinishSelector:@selector(partRequestDidFinish:)];

        // Store the object in the UserInfo dictionary
        NSMutableDictionary *userInfo = [NSMutableDictionary dictionaryWithObjectsAndKeys:obj, @"Object", nil];
        [request setUserInfo:userInfo];

        // Add it as a dependency
        [processBlock addDependency:request];

        // Add it to our network queue
        [networkQueue addOperation:request];
    }

    // Add the processBlock to our worker queue
    [workerQueue addOperation:processBlock];
}

Then you'll also need to write the delegate methods, the didFinish one would look something like this...

- (void)partRequestDidFinish:(ASIHTTPRequest *)request {
    // Remember this is one the main thread, so any heavy lifting should be 
    // put inside a block operation, and queued, which will complicate the 
    // dependencies somewhat, but is possible.

    // Get the result data
    NSData *data = [request responseData];

    // Get the object that it belongs to from the user info dic
    Object *obj = [[request userInfo] objectForKey:@"Object"];

    // Keep track of the partial data in the object
    [obj storePartialDataForPostProcessing:data];
}

And all of that would go into your class which connects to your server and creates your objects, so it's not a view controller or anything, just a regular NSObject subclass. It will need to have two queues, a managed object context (and more than likely a method which returns another MOC for you to use in threads, something like this:

// Fends a MOC suitable for use in the NSBlockOperations
- (NSManagedObjectContext *)moc {
    // Get a blank managed object context
    NSManagedObjectContext *aContext = [[UIApplication sharedApplication] managedObjectContext;
[aContext setMergePolicy:NSMergeByPropertyStoreTrumpMergePolicy];
    NSNotificationCenter *nc = [NSNotificationCenter defaultCenter];
    [nc addObserver:self selector:@selector(mergeChangesFromMOC:) name:NSManagedObjectContextDidSaveNotification object:aContext];
    return aContext;

}

- (void)mergeChangesFromMOC:(NSNotification *)aNotification {
    @try {
        [self.managedObjectContext mergeChangesFromContextDidSaveNotification:aNotification];
        NSNotificationCenter *nc = [NSNotificationCenter defaultCenter];
        [nc removeObserver:self name:NSManagedObjectContextDidSaveNotification object:[aNotification object]];      
    }
    @catch (NSException * e) {
        NSLog(@"Stopping on exception: %@", [e description]);
    }
    @finally {}
}

You'll also need to hook in some way of monitoring progress, re-queuing failed downloads, cancelling, and saving the MOC at the end. Re-queuing failed downloads is quite tricky. Anyway, hope that helps.

So, just to clarify, in your delegate method, you'd store the downloaded image in a temporary instance variable on your Object. Then when all 15 dependencies finish, you can access that instance variable and do your work.

---

To kick things off, 1 Queue should be enough for all the image requests.

What you might want to do is keep references to the request so you can cancel them if the object is no longer needed.

For the locking, there's a few things to consider with images:

1. They are compressed, so they need to be inflated before being UIImages, that is very heavy on the CPU.
2. If you ever want to write to the filesystem, that process is locking. Do it in another Queue to avoid locking.
3. It's never a good idea to store Blob into CoreData, store the file path as a string in Core Data and fetch it from disk using a queue

Just using 3 different NSOperationQueue will make your app much more responsive: 1 for ASIHTTPRequests (Don't create a new one, use the default with startAsynchronous) 1 for image writing to disk 1 for image fetching from disk

---

Since you will display image to view, why don't you SDwebImage:

SDImageCache manages an asynchronous download queue, ties the downloader with the image cache store, maintains a memory cache and an optional disk cache. Disk cache write operations are performed asynchronous so it doesn't add unnecessary latency to the UI.

[imageView setImageWithURL:[NSURL URLWithString:@"http://www.domain.com/path/to/image.jpg"] placeholderImage:[UIImage imageNamed:@"placeholder.png"]]

https://github.com/rs/SDWebImage