How can I get WebJob logs programmatically?

We created an Azure WebJob to perform a scheduled database cleanup for our WebAPI project. We now want to display the latest jobs in our own web management app to keep track of how the cleanup is going every day. How can I get the last 50 function calls including their respective output and duration logs for a given website?

Some prerequisites for this requirement:

The cleanup process we have encountered is very simple, but we suspect it will begin to see us in the near future, so we would like to track how long it takes to proactively rewrite it every day, using a more scalable approach when needed. Ideally, I would like to get the last 50 or so, and create a graph showing the time it took to complete during that time period.

The first thing we thought about was just creating our own database and transferring each execution with Stopwatch

, then storing the duration in our database. Then we could just query this database and graph that way.

But since this was the first time we used WebJobs, we quickly found out that most of the stuff we wanted to register was already automatically registered with the WebJobs SDK. Things like start time, duration, function name, etc. all already exist in the logs and it will be enough for us to create our own screen around.

The problem is how to request these logs from our MVC project in a suitable format. I could of course read from the blob store directly and manually merge the data, but I was looking for a higher level API. For me, the storage containers used by WebJobs are implementation details that may change in the future.

While searching for this, I have found numerous ways to get to the data, but I feel that none of them are what I was looking for.

• There is a Get-AzureWebsiteLog
  
  PowerShell cmdlet that returns data for a set of runs for a job. It looks ok, but I wanted something to call directly from .Net, and I would like to see the console outputs for this action too, and I cannot find them using this cmdlet.

• There is a WebAPI for querying logs for jobs. This seems to return the same response as the PowerShell version (the PowerShell cmdlet is probably calling the same REST endpoint under the covers). Thus, the same problems apply.

By examining the actual files generated on the blob storage account, I can access the output logs for each run by checking the folder /azure-webjobs-hosts/output-logs

and matching the id there with the values ​​in the folder /azure-webjobs-dashboard/functions/instances/

, so I suspect it would be possible to get them somehow using the API higher level. Also, if such an API were available, I wouldn't have to worry about creating secure model types for every API response and the like.