Accessing Jobsub job logs through the browser

Intended for:

Jobsub users.

Scenario/Use case:

The Jobsub wrapper script copies the output logs (stdout, stderr, etc) from completed jobs to scratch dCache, which are available to view through a web browser for approximately 20-30 days (dCache deletes least-recently-used files as needed to free up space). Links to these logs for a specific job can be found in Fifemon. These instructions will allow you to view logs from your own or a colleague's (depending on collaboration) jobs.

Instructions:

Locate your job in Fifemon (or if you already have a link to the Job logs, from POMS, email, etc, proceed to the next step):

1. Select your user name in the User Batch Details or User Batch History dashboard.
   
   
2. Click on the job ID in the "Job Clusters" table. You may have to narrow down the time range to when your jobs finished using the time picker in the upper right corner.
   
   
3. This will take you to the Submission Summary dashboard for that job, which includes various statistics on the jobs. There should also be buttons near the top, labeled "View sandbox files" and "View job event log".
   
   
   1. Clicking "View sandbox files" will take you to the complete listing of log files.
      
      
   2. Clicking "View job event log" will take you to a Kibana dashboard from which you can view individual job details and filter jobs based on exit code and more. It also includes direct links to the stderr and stdout logs from each job.

If you get a "User authorization has failed" error from the server:

This can mean one of several things:

1. You are trying to view another user's jobs, and:
   1. You are not a member of the user's collaboration, or you do not have your account set up for the experiment. You can request computer accounts and access be set up for you (requires collaboration approval).
   2. Your collaboration has not elected to make job logs viewable to all users. If your collaboration wants this enabled, the liaison or computing coordinator should open a request through the Service Desk.
   3. If the job is owned by a production account (e.g. dunepro), you need to have the "Production" role to view the logs; You can request the production role through Service Now.
   4. You are not a Jobsub "superuser" for your experiment. To add a user as a superuser, the liaison or computing coordinator should open a request through the Service Desk.
      
      
2. Another error has occurred. Please contact the Service Desk and open a ticket to Distributed Computing Support, providing the complete error message.

See Also:

Fifemon