开发者

Terrible cpu-time and ApiProxy$CancelledException

开发者 https://www.devze.com 2023-03-29 13:07 出处：网络
What I have done is a movie website. It will show the latest movie info and showtimes. I have a cron job to update all the playing movies. However, whenever it runs, it throws a ApiProxy$CancelledExce
相关专题：

What I have done is a movie website. It will show the latest movie info and showtimes.

I have a cron job to update all the playing movies. However, whenever it runs, it throws a ApiProxy$CancelledException and have a terrible cpu-time about 4706364 cpu_ms 4644739 api_cpu_ms that make my APP exceeds quota very soon. Could someone help me. Thanks!!

The number of the playing movies is around 25, and the number of the showtimes is around 650.

My code

// Get all the playing movies from the datasotre
Query query = pm.newQuery(Movie.class);
query.setFilter("playing == true");
List<Movie> playingMovies = (List<Movie>) query.execute();

// Update every playing movie
for (Movie m : playingMovies) {
    // getMovie() will return a movie with latest movie info and showtimes
    Movie leMovie = getMovie(m.getId());
    leMovie.setKey(m.getKey());
    leMovie.setFans(m.getFans());
    // because leMovie has the latest showtimes so I need to delete the older showtimes
    m.getShowtimeList().clear();

    pm.makePersistent(leMovie);
}

My Movie Class

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Movie {


    @Persistent
    private Set<Key> fans;

    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;

    @Persistent
    private String id;

    @Persistent
    private boolean playing;

    @Persistent
    private int gate;

    @Persistent
    private String picUrl;

    @Persistent
    private String mainName;

    @Persistent(mappedBy = "movie")
    @Element(dependent = "开发者_如何学编程true")
    private List<Showtime> showtimeList;

    //......

The Exception

2011-08-22 10:30:00.138
/cron/update-movie
com.google.apphosting.api.ApiProxy$CancelledException: The API call datastore_v3.Put() was explicitly cancelled.
    at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:298)
    at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:296)
    at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:67)
    at com.google.appengine.api.datastore.FutureHelper$CumulativeAggregateFuture.get(FutureHelper.java:144)
    at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
    at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
    at com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:113)
    at com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:110)
    at com.google.appengine.api.datastore.TransactionRunner.runInTransaction(TransactionRunner.java:31)
    at com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:110)
    at com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:94)
    at org.datanucleus.store.appengine.RuntimeExceptionWrappingDatastoreService.put(RuntimeExceptionWrappingDatastoreService.java:94)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:180)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:139)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:134)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.updateObject(DatastorePersistenceHandler.java:536)
    at org.datanucleus.state.JDOStateManagerImpl.flush(JDOStateManagerImpl.java:4576)
    at org.datanucleus.ObjectManagerImpl.flushInternal(ObjectManagerImpl.java:2814)
    at org.datanucleus.ObjectManagerImpl.flush(ObjectManagerImpl.java:2754)
    at org.datanucleus.ObjectManagerImpl.preCommit(ObjectManagerImpl.java:2893)
    at org.datanucleus.TransactionImpl.internalPreCommit(TransactionImpl.java:369)
    at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:256)
    at org.datanucleus.ObjectManagerImpl.close(ObjectManagerImpl.java:801)
    at org.datanucleus.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:271)
    at com.mm.servlet.UpdateMovieServlet.doGet(UpdateMovieServlet.java:100)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:97)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:249)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135)
    at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:262)
    at com.google.apphosting.base.RuntimePb$EvaluationRuntime$2.handleRequest(RuntimePb.java:9819)
    at com.google.net.rpc.impl.RpcUtil.handleRequest(RpcUtil.java:445)
    at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:414)
    at com.google.net.rpc.impl.Server$RpcTask.runInContext(Server.java:579)
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:454)
    at com.google.tracing.TraceContext.runInContext(TraceContext.java:694)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:332)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:324)
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:452)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:636)
C 2011-08-22 10:30:00.144
Uncaught exception from servlet
com.google.apphosting.api.ApiProxy$CancelledException: The API call datastore_v3.Put() was explicitly cancelled.
    at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:298)
    at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:296)
    at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:67)
    at com.google.appengine.api.datastore.FutureHelper$CumulativeAggregateFuture.get(FutureHelper.java:144)
    at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
    at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
    at com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:113)
    at com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:110)
    at com.google.appengine.api.datastore.TransactionRunner.runInTransaction(TransactionRunner.java:31)
    at com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:110)
    at com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:94)
    at org.datanucleus.store.appengine.RuntimeExceptionWrappingDatastoreService.put(RuntimeExceptionWrappingDatastoreService.java:94)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:180)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:139)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.put(DatastorePersistenceHandler.java:134)
    at org.datanucleus.store.appengine.DatastorePersistenceHandler.updateObject(DatastorePersistenceHandler.java:536)
    at org.datanucleus.state.JDOStateManagerImpl.flush(JDOStateManagerImpl.java:4576)
    at org.datanucleus.ObjectManagerImpl.flushInternal(ObjectManagerImpl.java:2814)
    at org.datanucleus.ObjectManagerImpl.flush(ObjectManagerImpl.java:2754)
    at org.datanucleus.ObjectManagerImpl.preCommit(ObjectManagerImpl.java:2893)
    at org.datanucleus.TransactionImpl.internalPreCommit(TransactionImpl.java:369)
    at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:256)
    at org.datanucleus.ObjectManagerImpl.close(ObjectManagerImpl.java:801)
    at org.datanucleus.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:271)
    at com.mm.servlet.UpdateMovieServlet.doGet(UpdateMovieServlet.java:100)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:97)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:249)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135)
    at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:262)
    at com.google.apphosting.base.RuntimePb$EvaluationRuntime$2.handleRequest(RuntimePb.java:9819)
    at com.google.net.rpc.impl.RpcUtil.handleRequest(RpcUtil.java:445)
    at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:414)
    at com.google.net.rpc.impl.Server$RpcTask.runInContext(Server.java:579)
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:454)
    at com.google.tracing.TraceContext.runInContext(TraceContext.java:694)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:332)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:324)
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:452)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:636)
W 2011-08-22 10:30:00.451
A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104)

Maybe it is too late to reply...

Usual request time is about one minute, if "getMovie(m.getId());" request info from some other server it may take long and go through your minute.

GAE task has more time to run jobs so better you use your cron to enqueue a new task and let appengine do your work in the background. 

Task t = new Task();
            t.setUrl(URL_SERVLET_TO_UPDATE_MOVIES);

            //submit task
            TaskReference tr = queue.add(t);

This servlet runs the job of updating movies.

Even needing more time (lets assume you got plenty of movies there) you can get only a few number of movies per query, actualize them, get a Cursor (works as range) and enqueue a new task with that cursor as parameter to the servlet. This new task will start from the next movie you have to actualize (thanks to the Cursor) and you will have full time to do your job. You can chain task until your job is done.

At your servlet look for a cursor as param:

String encodedCursor = req.getParameter(CURSOR_PARAM);
        if (  encodedCursor != null){
            oldCursor = Cursor.fromWebSafeString(encodedCursor);
        }

You can use this Cursor to query from the last movie you queried. Get a new cursor from the new query and add it as parameter to the new task.

Task t = new Task();
            t.setUrl(PROCESSOR_SERVLET);

            //add cursor as parameter
            t.addParameter(CURSOR_PARAM, cursor.toWebSafeString());

            //submit task
            TaskReference tr = queue.add(t);

Using Task and cursors you will get your job done in the background avoiding time limits. Of course, there are quotas over there...

Doc about cursors: https://developers.google.com/appengine/docs/java/datastore/queries#Query_Cursors

0

上一篇:

:下一篇

更多 问答 相关资讯：

精彩评论

暂无评论...
登录 注册
Qedev.com
验证码 换一张
取 消

    关注公众号

    热门标签

    图文推荐

    Delphi - Custom drawing a message list

    Delphi - Custom drawing a message list
    C++ header-only include pattern

    C++ header-only include pattern
    IE7 Margin Collapses Into Padding

    IE7 Margin Collapses Into Padding
    in CoffeeScript, how can I use a variable as a key in a hash?

    in CoffeeScript, how can I use a variable as a key in a hash?
    Interactive visualization of a graph in python [closed]

    Interactive visualization of a graph in python [closed]
    How to customise PHP MYSQL tables?

    How to customise PHP MYSQL tables?
    High quality, simple random password generator

    High quality, simple random password generator
    Image Recognition ApI in android

    Image Recognition ApI in android

    开发者开发者网给大家分享系统运维,大数据运维,云计算,编程开发技巧,路由交换,运维和开发相关的资讯及技术文章，同时StackOverflow中文社区，知识经验交流分享。

    法律声明：本站内容均为网友上传，网站举办方负责审核和监督，如存在版权或非法内容，欢迎举报，我们将尽快予以删除。邮箱：devze@qq.com

    Copyright © 2018-2020 开发者. All rights reserved. Powered by 开发者ICP备案号: 京ICP备10032868号-9