Forums/General

Any limits on the data returned from cloud.info?

carl
posted this on Jul 27 07:28

I am considering building an internal tool for displaying the output of longer running jobs.  The problem is that error logging to stdout / stderr might be large.  Are there any internal limits to the amount of data that is returned from calling cloud.info?

 

Comments

User photo
Aaron Staley
PiCloud, Inc.

Hi Carl,

We currently limit output to the last 1,000 lines written. See http://docs.picloud.com/client_adv.html#read-output for more information.

To get around this limit, set sys.stdout and sys.stderr to open files.  If you write to ~/shared/afile you will always be able to read from that file in the future.

e.g. In your logging job, start with:

sys.stdout = open('shared/stdout.%s' % uniqueid, 'w')
sys.stderr = open('shared/stderr.%s' % uniqueid, 'w')

 

Then you can read those files as follows:

def readFiles(id):
output = fopen('shared/stdout.%s' % id)
error = fopen('shared/stderr.%s' % id)
return output.read(), error.read() 
jid = cloud.call(readFiles, the_unique_id_you_used) 
stout, stderr = cloud.result(jid) 

You could also write these to cloud.files in a similiar way.

Please let us know if this technique resolves your concerns and if you have any further questions.

Thanks,

Aaron Staley

PiCloud, Inc.

Jul-27 2010 14:28.
User photo
carl

cool.  thanks.

Jul-28 2010 13:04.