Python API

API v310.64 Documentation Available

Greetings,

The v310.64 API documentation is now available on Tintri’s GitHub site. So what’s new:

  • Four new systemProperty APIs which allows customization of the following system properties:
    • used percent alert low threshold
    • used percent aler high threshold
    • reserves used percent alert threshold
    • use percent disable snapshots threshold
    • use percent disable replication threshold
  • more Datastore statistics
    • snapshot space saving factor
    • space savings factor including snapshot savings

For more explanations on system properties, go to the systemProperites Data Transfer Object (DTO) documentation; and for more information on the new snaphot factors, go to the DatastoreStat DTO documentation.

Currently the PowerShell Toolkit and the Python SDK do not support the system property APIs except in raw API form.  See SysPropertyManger.ps1 for an example. However, the new information in the Datastore DTO is available to the Python SDK and the PowerShell Toolkit.

Happy coding,

– Rick –

Advertisements

VM Scale-out Code Examples

There is a new blog about automating VM scale-out migration recommendations on the corporate Tintri site. Along with that script I wrote another script to set VM affinity for VM scale-out migration rules. These rules allow you to exclude VMs from being considered for migration.

Let’s look at a use case. Suppose there is one Flash VMstore, and 2 hybrid VMstores in a VMstore pool. VDI is running in your Flash VMstore, and you don’t what the VDI VMs to migrate to the hybrid VMstores, therefore, the VDI VMs are put into a Service Group by name pattern. Unfortunately, the script would have to be executed periodically because affinity is set at the VM level.  When a VM is added to the Service Group, the script needs to be executed.

VMs are excluded with --affinity set to ‘never‘.  VMs are specified by a service group, --sg, by a list of VM name, --vms, or by pattern matching with --name. To clear the affinity set --affinity to ‘clear‘.

So let’s look at the script, set_reco_vm_affinity.py.

# Let's get to work
try:
    if (len(vms) > 0):
        print("Collecting VMs from list")
        vm_uuids += get_vms_in_list(server_name, session_id, vms)

    if (service_group != ""):
        print("Collecting VMs from service group")
        sg_uuid = get_sg_by_name(server_name, session_id, service_group)
        if (sg_uuid == ""):
            raise tintri.TintriRequestsException("Can't find service group " + service_group)

        vm_uuids += get_vms_by_sg(server_name, session_id, sg_uuid)

    if (vm_contains_name != ""):
        print("Collecting VMs from name")
        vm_uuids += get_vms_by_name(server_name, session_id, vm_contains_name)

    if (len(vm_uuids) == 0):
        raise tintri.TintriRequestsException("No VMs to set rules")

    if debug_mode:
        count = 1
        for uuid in vm_uuids:
            print(str(count) + ": " + uuid)
            count += 1

    # Process according to affinity
    if (affinity == "never"):
        set_vm_affinity_never(server_name, session_id, vm_uuids)
    elif (affinity == "clear"):
        clear_vm_affinity(server_name, session_id, vm_uuids)
    else:
        raise tintri.TintriRequestsException("Bad affinity rule: " + affinity)

except tintri.TintriRequestsException as tre:
    print_error(tre.__str__())
    sys.exit(4)
except tintri.TintriApiException as tae:
    print_error(tae.__str__())
    sys.exit(5)

In the above snippet, the code figures out the list of VMs and processes for the specified affinity.

The crux of this script is how to set the affinity:

# A helper function  that sets the VM affinity rule for migration recommendations.
def set_vm_affinity(server_name, session_id, vm_uuids, affinity_rule):
    url = "/v310/vm/"

    for vm_uuid in vm_uuids:
        rule_url = url + vm_uuid + "/affinity"

        r = tintri.api_put(server_name, rule_url, affinity_rule, session_id)
        if r.status_code != 204:
            tintri.api_logout(server_name, session_id)
            message = "The HTTP response for put affinity rule to the server is not 204."
            raise tintri.TintriApiException(message, r.status_code,
                                            rule_url, str(affinity_rule), r.text)
        sys.stdout.write(".")
    print("")

# Set the VM affinity rule to never for a list of VMs.
def set_vm_affinity_never(server_name, session_id, vm_uuids):
    print("Setting " + str(len(vm_uuids)) + " VMs to never migrate")

    affinity_rule = \
        {"typeId" : beans + "vm.VirtualMachineAffinityRule",
         "ruleType" : "NEVER"
        }

    set_vm_affinity(server_name, session_id, vm_uuids, affinity_rule)

# Clear the VM affinity rule for a list of VMs
def clear_vm_affinity(server_name, session_id, vm_uuids):
    print("Clearing " + str(len(vm_uuids)) + " VMs affinity rules")

    affinity_rule = \
        {"typeId" : beans + "vm.VirtualMachineAffinityRule",
        }

    set_vm_affinity(server_name, session_id, vm_uuids, affinity_rule)

In the function set_vm_affinity(), we see the API invoke /v310/vm/{vm_uuid}/affinity. For each VM UUID in the input list, the API is invoked. There are two helper functions set_vm_affinity() and clear_vm_affinity to facilitate.

Although it has been discussed before, you can review how to get the VMs in a service group with this snippet of code.

# Return a list of VM UUIDs based on a service group name.
def get_vms_by_sg(server_name, session_id, sg_uuid):
    vm_uuids = []
# Get a list of VMs, but return a page size at a time
vm_filter = {"includeFields" : ["uuid", "vmware"],
             "serviceGroupIds" : sg_uuid
            }
vm_uuids = get_vms(server_name, session_id, vm_filter)

return vm_uuids

# Return VM items constrained by a filter.
def get_vms(server_name, session_id, vm_filter):
vm_uuids = []

# Get a list of VMs, but return a page size at a time
get_vm_url = "/v310/vm"
vm_paginated_result = {'next' : "offset=0&limit=" + str(page_size)}

# While there are more VMs, go get them, and build a dictionary
# of name to UUID.
while 'next' in vm_paginated_result:
    url = get_vm_url + "?" + vm_paginated_result['next']

    r = tintri.api_get_query(server_name, url, vm_filter, session_id)
    print_debug("The JSON response of the VM get invoke to the server " +
                server_name + " is: " + r.text)

    # For each VM in the page, print the VM name and UUID.
    vm_paginated_result = r.json()
    print_debug("VMs:\n" + format_json(vm_paginated_result))

    # Get the VMs
    items = vm_paginated_result["items"]
    for vm in items:
        vm_uuids.append(vm["uuid"]["uuid"])

return vm_uuids

Before I close this post, I would like to mention that you can post API questions on Tintri’s Hub. Until next time,

– Rick –

New Python API Examples

Greetings,

There are 3 updates to the Python API examples on GitHub. One is about updating data IP addresses on a VMstore.  The other 2 examples are on VMstore appliance information collection.

Updating  Data IP Addresses

I wrote an Python example, set_data_ip.py, that displays and/or updates the data IP addresses for a VMstore.  The script always displays the current IP addresses.  Depending on the options, one data IP address is added or removed. The main crux of the code is similar to the code setting DNS. This is described in  more detail in this week’s “Ask Rick” blog.

Collecting VMStore Information

I had previously added an example, appliance_info.py, which displays the following:

+-------------+--------------------------+
| Info        | Value                    |
+-------------+--------------------------+
| Product     | Tintri VMstore           |
| Model       | T445                     |
| All Flash   | False                    |
| OS version  | 4.2.0.1-7524.41058.18773 |
| API version | v310.51                  |
+-------------+--------------------------+

I have moved this code to appliance_status.py, because it displays short concise VMstore or TGC status.

Now that we have appliance_status.py, I re-created appliance_info.py to display the following information:

  • VMstore status
  • Appliance components
  • Failed components if any
  • IP addresses: admin, data, and replication
  • Controller status
  • Disk information

This script imports the prettytable module.

Until next time,

– Rick –

Adding VMs to a Service Group

Greetings,

A colleague, Adam  Cavaliere, wrote a Python script that adds VM name from a CSV file to a TGC Service Group. The script, set_service_group_members.py is now on Tintri’s GitHub site.

This script asks for the user name and password interactively instead being on the command line.  The command line only has the TGC server name, the TGC service group to put the names into, and the CSV file name that contains the VM names.

Let’s look at the salient part of the code.

if service_group_exists:
    target_sg_api = "/v310/servicegroup/" + target_sg + "/members/static"
    print("Service Group Found")

payload = {'typeId':'com.tintri.api.rest.v310.dto.CollectionChangeRequest', \
           'objectIdsAdded': uuid_list
}

# Set the static members.
tt.api_put(server_name, target_sg_api, payload, session_id)

Here we have the API invoke to update the static members of the specified service group with a list of VM UUIDs.  The list of VM UUIDs are added to payload before the api_put() call.

The code below obtains the target_sg for the update service group static members API URL  It performs a GET servicegroup and looks for the specified service group and extracts the service group UUID.

# Get List of Service Groups
api = "/v310/servicegroup"
results = tt.api_get(server_name, api, session_id)
service_groups = results.json()

service_group_exists = False

# setup API for adding VMs to proper Service Group
for item in service_groups["items"]:
    if item["name"] == service_group:
        service_group_exists = True
        target_sg = item["uuid"]["uuid"]
        break

From a list VM names, the VM UUIDs are collected in the get_vm_uuids() function. This function takes as input a list of VM names and returns a list of found VM names and their UUIDs.  The GET vm API is is used.  I have discussed this in previous blogs. The UUID list is used in the payload above.

That pretty much covers it,

– Rick –

GitHub API Python Examples Updated

Greetings,

The API python examples on the Tintri GitHub site have been updated.

The library, python_1_1.py has been updated to disable SSL warnings. To facilitate this, a new method, api_version(), has been added to the library. It replaces the /info GET invoke.

All the API python client examples have been updated to use the python_1_1.py library and api_version() method. Since api_version() returns the same values as /info, the change is minimal. Here is an example diff:


 - r = tintri.api_get(server_name, '/info')
 + r = tintri.api_version(server_name)
      json_info = r.json()
      product_name = json_info['productName']

A new API client example, set_qos_tgc_service_groups.py is now available. This example shows how to use the new QoS TGC Service Group APIs. With the new APIs, configuring QoS for Service Groups has been simplified. The blog, TGC Service Groups and QoS Redux discusses this new QoS example.

Regards,

– Rick –

Some API differences with Virtually Aware Flash Storage

Greetings,

Tintri has just announce the T5000 All-Flash series.  This means that customers that desire all-flash storage now can have it all: flash, and virtually aware storage (VAS).  I recently blogged about some API and UI changes.

I wanted to go into a little more detail about the API changes.  The All-Flash code snippet in the blog is from an updated appliance_info.py. All the python files in the  Tintri API GitHub repo were updated with a more common python location for easy script running.

As for dedupeFactor and cloneDedupeFactor in the DatastoreStat object discussion, I hope the tables below clarify.  The first table shows the responses when invoking the following APIs from a VMstore.

  • GET /v310/datastore
  • GET /v310/datastore/{uuid}/statsHistoric
  • GET /v310/datastore/{uuid}/statsRealtime
  • GET /v310/datastore/{uuid}/statsSummary
Responses from VMstore
Field All-Flash Hybrid (4.0) Hybrid (3.2)
dedupFactor Valid Valid Not Available
cloneDedupFactor 1.0 Valid Valid

As you know, Tintri Global Center (TGC) and VMstore have the share some above API endpoints, but the responses can be different. Here is the table for the TGC responses:

Responses from TGC
Field All-Flash Hybrid (4.0) Hybrid (3.2)
dedupFactor Valid Valid 1.0
cloneDedupFactor 1.0 Valid Valid

As you can see, invoking the APIs via a TGC masks the “Not Available” for a consistent return for all fields.

Until next time,

– Rick –

Setting DNS primary at Scale

Greetings,

Another “Ask Rick” was posted on Tintri’s corporate blog.  This post covers setting the VMstore’s primary DNS for 100s of VMstores.  This was done for our IT department.

The main thrust of this article, is how to modify VMstore configuration information with the Appliance API.  The Data Transfer Objects (DTOs) are hierarchical.  Let’s start with the Request DTO, because the Appliance API is PUT with it in the HTTP body.


# Create the Request object with the Appliance DTO.
Request = \
{'typeId': 'com.tintri.api.rest.v310.dto.Request',
 'objectsWithNewValues': newAppliance,
 'propertiesToBeUpdated': ['dnsConfig']
}

As you can see, the objectsWithNewValues field is an Appliance object, and the propertiesToBeUpdated is the dnsConfig field in the Appliance DTO since DNS configuration is desired to be changed.


# Create the Appliance object wit the new ApplianceDns DTO.
newAppliance = \
{'typeId': 'com.tintri.api.rest.v310.dto.domain.Appliance',
 'dnsConfig': newDnsInfo
}

Above we see the newAppliance object with the dnsConfig field only.  We don’t need the other Appliance fields since we are not modifying them.  The dnsConfig field contains the new DNS values we want to set.


# Create the ApplianceDns DTO.
newDnsInfo = \
{'typeId': 'com.tintri.api.rest.v310.dto.domain.beans.hardware.ApplianceDns',
 'dnsPrimary': new_dns_primary,
 'dnsSecondary': new_dns_secondary
}

Both primary and secondary DNS values need to be present because the API is modifying dnsConfig as a whole.

After the DTOs are created and filled with the appropriate values, the Appliance API is invoked.


APPLIANCE_URL = "/v310/appliance/default"

url = APPLIANCE_URL
r = tintri.api_put(server_name, url, Request, session_id)

This template will be useful for setting other values in the Appliance.

Please check out this week’s “Ask Rick” for more details.  As always, you can leave comments and questions here.

Regards,

– Rick –

QoS and TGC Storage Groups

Greetings,

My latest blog post on “Setting QoS at Scale” was posted on the Tintri corporate site.  This post discusses how to use TGC Service Groups to set storage QoS on multiple VMstores.  TGC Service Groups allow you to organize VMstores according to your needs.  For example, one approach might be for all your VMstores associated with VDI VMs, and another approach could be for VMstores associated with a specific customer.

For more information on storage QoS, check-out these two posts:

You’ll note that Tintri’s QoS satisfies Mr Crump’s three requirements:

  1. storage QoS at the VM level
  2. expressing IOPs the same for each VM independent of block sizes
  3. storage resource visibility

Combining TGC Service groups and setting storage QoS provides an great way to manage your VMstores.

– Rick –

Measuring Storage Usage the Easy Way

Greetings,

I posted a blog entry on the Tintri cooperate blog.  This is a change in the corporate blog style to allow more technical blog posts.  Please feel free to comment.

The blog entry discusses how to collect statistics for chargeback and/or showback.  A new collection statistics code example posted on our Tintri GitHub site is discussed in the blog post as well.

So check it out.,

– Rick –

Delete the Oldest Snapshot on a VMstore

Happy New Year,

For the beginning of the New Year, I have coded an REST API example in Python.  This example queries for the oldest user generated snapshot and removes it.  This example shows how to use a query string with the GET snapshot API invoke, and a DELETE snapshot.  What is cool about this example is that there are functions that encapsulate the requests Python modules calls that hopefully will be reusable.  I have attached the example, and will go through some of the code below.

First off, you will need these Python modules:

import requests
import json
import sys
from datetime import datetime

Let’s look at the login function.  A user name and password is all that is needed over HTTPS.

def api_login(server_name, user_name, password):
    # Payload, header and URL for login call
    headers = {'content-type': 'application/json'}
    payload = {'username': user_name,   # payload needed for post.
               'password': password,
               'typeId': 'com.tintri.api.rest.vcommon.dto.rbac.RestApiCredentials'}   # 'typeId' is required.
    url_login = 'https://'+ server_name + '/api/v310/session/login'

    try:
        # Invoke the login API.
        r = requests.post(url_login, data=json.dumps(payload),
        headers=headers, verify=False)
    except requests.ConnectionError:
        print_error("Login: API Connection error occurred")
        sys.exit(-2)
    except requests.HTTPError:
        print_error("Login: HTTP error occurred")
        sys.exit(-3)
    except requests.Timeout:
        print_error("Login: Request timed out")
        sys.exit(-4)
    except:
        print_error("Login: An unexpected error " + sys.exc_info()[0] +
                    " occurred")
    sys.exit(-5)

    # if HTTP Response is not 200 then raise an exception
    if r.status_code != 200:
        print_error("The HTTP response for login call to the server " +
                    server_name + " is not 200, but is: " + str(r.status_code))
        print_error("url = " + url_login)
        print_error("payload = " + str(payload))
        print_error("response: " + r.text)
        sys.exit(-6)

    session_id = r.cookies['JSESSIONID']   # Session ID is return for subsequent invokes.
    return session_id

The info API does not require user name, password, or session ID.

r = api_get(server_name, '/api/info')
json_info = r.json()

The query below requests that the response returns the oldest user generated snapshot.

# Create filter to get the oldest user generated snapshot
q_filter = {'queryType': 'TOP_DOCS_BY_TIME',
'limit': '1', # There can only be one
'type': 'USER_GENERATED_SNAPSHOT'}

# Get the oldest user generated snapshot
r = api_get_query(server_name, '/api/v310/snapshot', q_filter, session_id)

This example could be used for the basis of a snapshot delete program.  Additions could be added to the query string, like VM name or before this time.

The code example is here: delete_snapshot.

Regards,

– Rick –