Use the Promote UI to manage the models you've deployed.
You can view models on the Home page of the Promote UI. The page has three columns, which represent the different stages in the lifecycle of your model:
- The Dev column lists newly deployed models.
- The Staging column lists models that are under construction and ready for review.
- The Production column lists models that are in use.
To promote a model from one stage to another, select the down-arrow icon that corresponds with the model you want to promote. Select Promote from the dropdown menu.
To return a model from Staging to Dev, select the down-arrow icon that corresponds with the model you want to return. Select Reject from the dropdown menu.
Every time you deploy a new model, Promote archives the previous version, encrypts it, and persists it to disk.
You can only run one version of any given model at a time. This ensures that the online, named model is the only one that returns information to users.
Promote defines two types of models:
- User-level models
- In-production models
For either type of model, Promote assigns a route. This is the URL that users send POST requests to.
User-level models are models you can query for predictions to test whether the models are ready for production. These models are in the Dev or Staging columns in the Promote UI. They have routes that follow this pattern: /PROMOTEURL/USERNAME/models/MODELNAME.
In-production models are models you use to return predictions. All these models are in the Production column in the Promote UI. They have routes that follow this pattern: /PROMOTEURL/production/models/MODELNAME. An important part of managing models is separating those you have in development from those you have in production. Use that dedicated production route to separate user-level models from in-production models.
- # In this example, the USERNAME is "production," the MODELNAME is "HelloWorld," and PRODUCTION is TRUE.
If you select the name of your model from the Home page of the Promote UI, you see several sub-tabs:
The sections below explain what each of those tabs do.
The Overview tab provides general information about the model. The tab contains four subsections:
- The Info section provides you with this information about the model:
- API Endpoint
- The Code section displays the actual code for the model (only for Python models).
- The Objects section lists any objects the model uses (only for Python models).
- The Packages section lists the name of packages the model requires, as well as their corresponding versions.
The Execute tab provides a test area for the model. You can input code in the left field in JSON format, then select Execute Model. Results populate in the display on the right.
The Samples tab provides code to execute the selected model in various languages, including:
Select the tab that corresponds with the language you want to use, copy the code, and then paste the code snippet into your production environment.
The Versions tab provides version information about the model:
- Previous versions of the model, up to and including the current deployment
- Metadata information from when you deployed the model
- The status of your model
If you select a model from the Dev or Staging column, the Versions tab includes all previous versions. If you select a model from the Production column, the Versions tab only includes production versions of the model.
Use the Previous and Next buttons to see this information for the various versions of the model.
Select Redeploy to relaunch a previous version of the model.
When you redeploy a model, the current version of the model remains active until the version you want to deploy is ready. After that, Promote performs a zero-downtime switch to that version.
The Logs tab provides the most recent log output about the model from your cluster.
The Analytics tab acts as a monitoring dashboard, which provides information and graphs about the model.
Select a time range to visualize different time scales, from the last hour to the last 14 days. The aggregations and graphs update to match your selection. By default, the UI shows the last day.
The UI displays aggregations of key details:
- Requests shows the total number of requests made against the model.
- Failures shows the total number of requests that failed.
- Success Rate shows the percentage of successful requests out of total requests made.
- Avg Latency shows the average latency rate between requests made and responses returned.
The UI also displays graphs that show details about requests across time:
- Total Requests shows each request made against the model.
- Response Codes shows what response code the Promote cluster returned.
- Mean Latency shows the average latency in milliseconds.
The History tab provides a list of all predictions that users request from the model. It displays this information for the past 14 days or until the information reaches a 50 GB limit.
To set controls for prediction logging, see Admin Tools.
The table displays this information:
- The Promote ID column shows the unique ID that Promote assigns to each request.
- The Time column shows the time when the model processed the request.
- The Version column shows the version of the model that processed the request.
- The Response column shows what response code the model returned.
- 200 codes indicate successes.
- 300 codes indicate redirections.
- 400 codes indicate client response errors.
- 500 codes indicate server response errors.
- The Input/Output column contains the button Show. When selected, it displays a code container with the input data and output results from the model. To close the container, select Hide.
Use the Filters tool on the left side of the page to filter by Date, Response Code type, and Model Version. Select Reset All to remove any filters.
The Advanced tab contains two sections that allow you to control the environment of the model.
1. Base Image
See the base image for the model.
2. Number of Replications
Allocate resources to your model by using the controls for number of replications. Use the arrows to increase or decrease the number of replicas available to the model. Select Save to update the resources you've dedicated to the model.
3. Environment Variables
Select the + icon to set your environment variables.
To add an environment variable, input its name and value in the appropriate fields. Select Save.
To remove an environment variable, select the down arrow on the associated line. Select Delete from the dropdown menu.
If you add or remove an environment variable, Promote rebuilds the model.
You need to remove and create a new variable to update its value. Update the value of an existing environment variable by typing the new, desired value into the appropriate field. Select Save.
To restart the model, select Restart.
To redeploy the model, select Redeploy.
To delete the model, select Delete, then select Confirm in the confirmation window.