Skip to content
Vantiq Community
Sign In Register

Production Applications Best Practices

Skip to main content

Introduction

We have made a distinction between deployed applications that are operational and deployed applications that are production. In this paper we offer some best practices that can be applied to bring an application into compliance with the production application criteria enumerated in the document Deployed Applications: Operational and Production.

Practices for Producing Production Applications

Development and operations staff can apply the following criteria and methods to bring the quality of an operational application up to the standard of a production application. Certainly, all of these methods can be applied, but it is also reasonable to be more selective and only apply those methods required to bring the specific application of interest up to production standards.

Criteria and methods:

1Quality hygiene

  • Application produces no errors (aka exceptional conditions) during normal operation. If a normal run of the application produces errors in the error log, the application is not ready for production. Note that this does not apply to user errors that are properly handled and reported to the user; it only applies to errors that are not handled by the application.
  • All diagnostic logging is disabled. An application may produce a small number of log entries during normal operation, but if it is writing debugging and diagnostic information at more than one record every few seconds, the application is not suitable for production. Remember there are 86,400 seconds in a day, implying that an application that writes 10 log entries a second will produce nearly a million log records a day. That adds up, and you will quickly exceed your disk quotas.
  • A set of documented tests is available that confirms the proper operation of the application. It is most useful to have automated regression tests where practical, since these can then be run on demand when changes are made to the application for bug fixes or enhancements. The tests can also be run when new versions of the platform are introduced to confirm there is no impact on the correctness of the application.

2Scaling hygiene

  • The application does not produce errors when run at scale. Typically, the additional errors that occur at scale are quota violations caused by the application using more resources than it is authorized to use. The error logs can be scanned to detect such errors. If scaling tests are producing errors, the application is definitely not ready for production.
  • The application has been tested at scale. This does not necessarily mean it has been tested at full scale – although that is a good idea – but it has been tested with enough load that you are confident the production workload will be serviced properly.
    Tip: A by-product of scale testing is that you should know what resources are required to execute a unit of work within the application. Resources are normally measured as events ingested or produced and DBMS actions taken. These measures are proxies for the overall resource consumption of the application. This information can be used for capacity planning and to estimate the cost of running the application at greater scale. See the document Scalable Application Architectures for additional details.

3Code management in place

  • It must be possible to re-create the application from its source representation. This is a general rule for all software development. In VANTIQ, the source representation is the set of resources that comprise the application. All such resources should be exported and placed under the control of a version management system such as Git or Team Foundation Services. Any other resources required to reconstitute the complete application – for example, configuration files and help files – should also be placed under version control. Essentially, everything needed to recreate the application from scratch should be controlled.
  • The deployment procedure is documented and under version control, along with the application resources. If you are using the VANTIQ deployment manager, exporting the configurations, deployments and environments used to implement the deployment will satisfy this need. If you are using external procedures and scripts to deploy the application, they must all be collected and placed under version control, along with the instructions for executing the deployment process.

4Namespace and nodes provisioned

  • The application will run in one or more namespaces within the production installation. The namespaces must be set up in anticipation of deploying the application. Nodes can be defined as part of the deployment process, and the node definitions must also be placed under source management.

5User management in place

  • Users are provisioned into the production installation. The application is likely to have a user community unless it is a totally automated system. Even if users are provisioned into the development installation, they must be provisioned again in the production installation. The normal procedure is to invite them to be authorized users within the namespace in which the application executes.
  • Profiles that contain appropriate privileges for each user role are constructed and assigned to the appropriate users.

Learn more: Namespace and user provisioning for production applications is described in detail in Migration to API. That document describes a number of additional best practices for bringing applications to production quality.