Running scripts to process GIS data updates became more and more of a hassle as demand quickly grew. There needed to be a way to automate these tasks while managing users and workflows...
Las Vegas is quickly becoming one of the nation's largest innovation hubs. Having extensive partnerships with NTT Data, Cisco, and a multitude of self-driving startups, it is becoming the testing ground for smart city initiatives. Technology initiatives like this brings big data, and updating, processing, and monitoring that data started to become a bottleneck for the GIS team. This is why I created AutoGIS. AutoGIS is a cloud-based enterprise level job scheduling and monitoring tool built to manage data processing tasks. It provides a centralized way to automate workflows, manage user data, monitor server uptime, and share important updates.
AutoGIS provides a host of features to manage data processing workflows. First and foremost, it automates repetitive processing tasks by scheduling them based on a set time frame. Once a task is executed, statistics such as it's execution time, return code, and log data are sent to the cloud to provide a full searchable history of all executed jobs.
To fully automate our processes, specific workflows can be configured based off the results of an executed task. For example, when a task requires that another task be successfully executed before running, you can set a rule to schedule said task after the execution of the required task, but only if it executes without fail. If it does fail, you can schedule a different task to mitigate the failure, or set up an alert.
An alert is how AutoGIS keeps users and customers informed about important updates. Just like how tasks can be daisy chained to create specific workflows, alerts can be triggered based off a task result as well. For example, when a task successfully executes, you can set up an alert to automatically send an email and/or a text to those who rely on the data.
AutoGIS has been a huge success so far. We haven't publicly made this available to other GIS professionals yet, but within our group it has already scheduled and executed over 17,000 processing tasks in less than 9 months. If you take each task and assume just 5 minutes to manually find the script, run it, then check to validate it executed properly, and send an email to a coworker to let them know it finished executing properly, AutoGIS has already saved us over 1400 hours worth of time! Once we expand this out to other users and data processing updates become more and more of a hassle, I'm sure that number will rise exponentially!