We’ve had some interesting discussions on our team recently, about the level of testing required for some very declarative sections of our codebase. I’ve been thinking about this subject a lot, especially after reading a recent post by Kent Beck. The fundamental problem we were trying to solve was this: We have a Python dictionary that represents an entity–in this case, a person. All of its values are strings:
Continue reading

I’ve been working to add support in slap for publishing Geoprocessing (GP) services; the workflow outlined here will work with both Python Toolboxes as well as the older, binary .tbx file format. Similar to map services, it’s possible to publish geoprocessing services as part of a docker build; all of the scripts, tools, and dockerfiles used in this example are available on github: slap base docker images test image test data and scripts Publishing from a Result File The createGPSDDraft method takes either a Result object or a path to a result (.
Continue reading

A lot of smart people have worked to get ArcGIS Server (AGS) up and running in a docker container; it’s a fast and convenient way to test applications that require an AGS instance to function. It’s possible to streamline things even further by registering data and publishing services as part of the docker build process; this allows you to keep your infrastucture, data, and service definitions under source control, and use those to build an image with all the necessary services included.
Continue reading

I’ve spent a good deal of time helping teams adopt automated testing as a part of their workflow; one thing that I’ve often heard from folks who aren’t familiar with the methodology is the following: “How much time do you estimate for writing unit tests?” Usually what people are looking for is something like, “I need X hours for the production code, and Y hours for writing tests for this feature.
Continue reading

I’ve seen a lot of GIS developers struggle to create a good project structure when building Python applications; often there’s a transition from one enormous file with a single method to a “real” software project, with modular design, well defined dependencies, and the necessary tooling. The goal of this post is to be a summary and short checklist; these steps can improve almost any project, and are easy to implement.
Continue reading

Virtualenv allows you to create a repeatable, isolated environment for your project and its dependencies, without worrying about what packages and versions are installed globally on your development machine. This is a standard tool for most python projects, but since arcpy is installed as a separate, global package, using virtual environments is a little more difficult. There are a couple of approaches to tackling this problem; either adding a .
Continue reading

Test fixtures can help us run our each of our tests against a clean set of known data, but arcpy can still throw a few curve balls at us–there are global singletons we need to be aware of (i.e., arcpy.env), and many GP tools can have side effects (such as changing the current working directory) or limitations of their own (i.e., the Project and Copy tools don’t support in-memory workspaces).
Continue reading

Test Driven Development has been a major part of the discipline of software engineering for almost two decades; a Red-Green-Refactor cycle helps keep codebases maintainable, while lowering the risk of adding new features by protecting against regression and facilitating extensible designs. The design and history of arcpy and its related modules can make it difficult to test, and automation is not always given high priority in GIS projects. Nevertheless, there are techniques to help ease the process, and it’s relatively easy to test drive the development of GIS software.
Continue reading

As a full-time remote developer, it goes without saying that an internet connection is a requirement; every developer will be pushing and pulling from source control, chatting via slack, and performing hundreds thousands of package installs and/or stack overflow searches. I also spend a lot of time using some sort of screensharing tool–all of our production code is written in pairs over screenhero, and we do our daily standups, architecture discussions, and watercooler sessions over Google Hangouts.
Continue reading

Look at any modern application, ETL process, or data analysis, and you’re likely to find a textual component. While the ubiquity and value of textual information is clear, working with text requires specialized training and knowledge that often fall outside the realm of traditional developers and analysts. The power and insight that can be gained from textual data coupled with the unique skills needed have led to an explosion of demand in the text industry.
Continue reading