Cloudbolt integrates with Tintri


Clouldbolt, a hybrid cloud management system, integrated their GUI with Tintri by using Tintri’s Python SDK.  Watch this video by Tristan Todd.  As you can see in the video, Cloudbolt integrated Tintri’s snapshots, clones, and statistics. Cloudbolt’s graphs are similar to Tintri’s GUI graphs which make it easy to move between products.

If you’re interested in the integration, Cloudbolt provides the code on their GitHub site in the ui_extensions directory.  For the Tintri extension, look in the tintri directory and follow the directions.

As Tintri’s Developer Evangelist, what interests me in the product is that it used Tintri’s PySDK.  This is one of the reasons why PySDK was developed:  to facilitate integration of current IT management and automation products with Tintri storage.

Until next time,

– Rick –

Upgrade VMstores in Parallel


I just added some PowerShell code that upgrades VMstores in parallel to our GitHub site.  A JSON configuration file is used to configure the VMstores.  For example:

        "VmstoreDnsName": "vmstore1",
        "Username": "username",
        "Password": "password",
        "UpgradePathToFile": "C:\Users\TestUser\upgrade_file.rpm"
        "VmstoreDnsName": "vmstore2",
        "Username": "username",
        "Password": "password",
        "UpgradePathToFile": "C:\Users\TestUser\upgrade_file.rpm"

The only required attribute is the VmstoreDnsName. All the other attributes are optional and can be from the command line.  Here is a JSON configuration with only the VMstore names:

        "VmstoreDnsName": "vmstore1"
        "VmstoreDnsName": "vmstore2"

The following code can uses the above file.

Import-Module -Name .\VmstoreUpgrade.psm1
Import-Module -Name .\Parallel-Upgrade.psm1

# Set the user, VMstore configuration file, and upgrade file.
$user = "admin"
$vmstore_list_file = ".\vmstore_upgrade_list.json"
$upgradeFile = "C:\tmp\txos-"

# Do the parallel upgrade.
Write-Host "Parallel updating VMstores with $upgradeFile"

# Get credentials that will work for all VMstores.
$cred = Get-Credential -Username $Username -Message "Enter valid password for all servers"
$pass = $cred.GetNetworkCredential().Password

# Start the parallel upgrade using a JSON file containing the VMstores, and
# the same username and password for all VMstores. 
Parallel-Upgrade -Verbose -UserName $user -UseSameUserName -Password $pass -UseSamePassword -UpgradeFile $upgradeFile -UseSameUpgradeFile -UseJsonFile $vmstore_list_file

Write-Host "Done Updating"
Write-Host ""

The above code gets a credential that is valid for all VMstores in the configuration file. Then the command line specifies the user name, password, and two options, -UseSameUserName, and -UseSamePassword.

Parallel-Upgrade logs to a dated log file: YYYY-MM-dd-HH-mm-ss_parallel_upgrade.log. The Verbose option will produce extra verbiage to follow along in the process and assist in debugging problems.  Unfortunately, you have to read the Parallel-Upgrade.psm1 module for help.

Until next time,

– Rick –

Invoke-RestMethod Exception Handling with Tintri APIs


Recently, I’ve been working in PowerShell.  Tintri has a great PowerShell Toolkit, but unfortunately some APIs cannot be invoked via the Toolkit.  So to invoke APIs that are not in the Toolkit, you can use Microsoft’s Invoke-RestMethod.  In this blog, I’m only going to discuss exception handling.

Try {
    $resp = Invoke-RestMethod -Uri $url -Method Delete -WebSession $session -ContentType $JSON_CONTENT
Catch {
    # Obtain some information
    $expMessage = $_.Exception.Message
    $failedItem = $_.Exception.Source
    $line = $_.InvocationInfo.ScriptLineNumber

    # Check if there is a response.
    if ($_.Exception.Response -eq $null) {
        Write-Host "At $($line):`r`n$expMessage" -ForeGroundColor Red
    else {
        # Get the response body with more error detail.
        $respStream = $_.Exception.Response.GetResponseStream()
        $reader = New-Object System.IO.StreamReader($respStream)
        $respBody = $reader.ReadToEnd() | ConvertFrom-Json
        $errorCode = $respBody.code
        $errorMessage = $respBody.message
        $causeDetails = $respBody.causeDetails
       Write-Host "At line $($line):`r`n$expMessage`r`n$($errorCode): $errorMessage`n`r $causeDetails" -ForegroundColor Red

Take the example code above where we invoking a GET API.  The variable $session is the session ID and $url is of course the API URL.

If there is an Invoke-RestMethod() error, the exception is thrown and the Catch block is executed.  When a Tintri API error occurs Invoke-RestMethod does not return any data, and there is more information in the API response.  See the Error Handling section in the API documentation where the error response is documented:

 "message":"Failed to find requested resource of type VirtualMachine with id: 'xxxxddd'",
 "title":"Resource not found"

So how do you get that error information in the response?  Fortunately Chris Wahl shared some great information on his blog.  As you can see, the first three bold lines in the second part of  the Catch block is directly from Chris.  The italicized part is what I added for Tintri errors.

The code now checks Exception.response; therefore, this Catch block can now handle both non-Tintri and Tintri errors.  I changed Write-Error to Write-Host with a red foreground color for simpler output.

The response stream returns a string.  To get the error code and the error message, the split function would need to be used twice within a loop; or just call ConvertFrom-JSON which which gives us a nice dictionary to easily pluck out errorCode, errorMessage, and causeDetails.

Until next time,

– Rick –


System Center Orchestrator Example


My colleague, Matt Geddes, has create a new 3 part blog on adding a script to System Center Orchestrator.  As usual, Matt sets up the problem in part 1. In part 2, he shows how to add a script to System Center.  And finally in part 3, you get your hands dirty with the actual PowerShell SyncVm script.  Matt has nicely provided a way to test in a stand-alone environment before it is added to System Center.

A new directory, SystemCenterOrchestrator in Tintri’s PowerShell GitHub repo has been added which contains the Sync VM example discussed in the above mentioned blog.

If you desire more SyncVM information, please check out these posts.

Now go plug into System Center,

– Rick –

V310.61 API Documentation Available


New Tintri API documentation is now available for version v310.61. Version v310.61 maps to VMstore‘s TXOS 4.3 and TGC 3.5 which contains additional APIs to support Synchronous Replication and disk encryption.

There is a new GitHub repository, tintri-rest-api, which contains the documentation and code examples. The repository tintri-api-examples is now deprecated.

Documentation for older API versions are still available on Tintri’s support site.

Until next time,

– Rick –

Private Cloud Automation

I want to point out a set of 4 postings by Matthew Geddes, a Hyper-V architect, on Automation and Private Cloud.

  1. Part 1: sets the stage: a number of production SQL Server instances where some nightly reporting jobs are run. These reporting jobs are quite heavyweight and impact other users during their large processing window. What we want to do is to have the reporting jobs run on non-production VMs with that day’s production data.
  2. Part 2: shows how how to use syncVM from Tintri’s PowerShell Toolkit.  Matt breaks down usage of the PowerShell cmdlet, Sync-TintriVDisk; and then shows how to integrate it into a small script.
  3. Part 3: adds more code to the small script and wraps it into a function so that it can be used for multiple reports.
  4. Part 4: adds more automation with by showing how to parameterize the reports.  Matt summarizes and gives you some homework.

I like the way Matt goes through presents the use case, and then goes through the code step by step.

Until next time,

– Rick –

Tintri vSphere Web Client Plugin


Just a quick note about the Tintri vSphere Web Client Plugin which has been released as GA. It’s now available for download on Support Portal. This is a maintenance release that includes support for the following key features:

  • SyncRepl functionality: support is available for VMstores with synchronous replication.
  • vSphere Plugin version:  displays the plugin version number.
  • Snapshot display: snapshots older than seven days are displayed.
  • IP/FQDN settings: change vCenter server IP/FQDN from the vCenter settings.
  • VM protection:  provides a Snapshot and replicate every minute option.
  • vCenter support:  supports vCenter 6.5.

Here is a list of bugs that have been fixed:

Bug # Description
52631 Customer may be unable to use SyncVM refresh
50475 Customer may be unable to use the ESXi Best Practice if the host is in a non-responding state
45710 Web Client Plugin may not load after upgrading vCenter to vCenter 6.0u2
45617 Snapshot clone behavior in the Web Client Plugin may be different from the VMstore UI when multiple datastores are involved
45049 VMstore may flood vCenter with alarm definitions
43929 Tintri IOPS, Tintri Throughput and Tintri Latency graphs may not display in the datastore Summary tab
43703 The delete snapshot permission may not be available for specific user roles within the Web Client
42901 The Web Client Plugin may hang when customers attempt to refresh a VM disk

After going to the support download page, search for “Tintri vSphere” for the new download.


– Rick –

Synchronous and 1-to-many Replication


I usually blog about APIs, but today I want to point to 2 excellent videos showcasing Tintri’s Tintri Global Center (TGC) GUI. One is on Synchronous Replication and Failover; and the second one is on 1-to-many replication.

Tomer Hagay first reviews the Synchronous Replication configuration which uses Tintri’s Service Groups. Next, he shows all the statistics that are available; for example, the VM latency components: host, network, contention, flash or disk, and mirror. Finally, Tomer shows how easily it is to do transparent failover.

Bill Roth first demonstrates how to find the VMstores’ replication information. Next, he shows how to get protection information for VMs.  How to add a new replication link to a VM is shown. A VM can replicate up to 4 different destinations. Of course Service Groups can have 1-to-many.replication too. As you know, one of the cool things about Tintri, is that Tintri has statistics; and in this case, Bill shows the two replication statistics that are available:  MBytes Remaining to to be replicated, and Replication Rate, both logical and network.

So check out these informative videos and enjoy Winter Solstice,

– Rick –


Tintribot Is Here to Help


As you might recall my post on Using Tintri APIs to Build Tintri Anywhere discussing using Slack as a management interface.  We now have 2 videos on Tintribot as an example. Please note that this is a possible future.

The first video, Manage storage with ChatOps from Tintri between bites, features an admin, Krystle, approving a recommendation from our VM Scale-out feature.

In the above video, Tintribot has code that can read VM Scale-out recommendations and approve them which is covered in this Tintri post. More VM Scale-out code can be found in this post.

The second video, Manage storage from anywhere with Tintri ChapOps, has admin Krystle bringing up VMs and putting production data on them..

First Tintribot brings up 500 VMs using Tintri cloning. Then the bot uses SyncVM to sync the 500 newly created  VMs to the latest recovery point of the production SQL data.

So how do you manage your storage?

–  Rick –

Changing QoS during Veeam Backups

Tintri’s QoS is a great tool to limit storage bandwidth per VM, but what happens when a backup occurs?  Well, QoS will also limit your backup storage IOPS. If this is an issue in your environment, you might consider clearing the QoS during backup.

As an example of clearing QoS when a backup occurs,  a PowerShell script is now available that interacts with Veeam Backup and Replication.  The script, Veeam_Backup_Tintri_QoS.ps1 is available on GitHub.  The file, has the information needed to run the script.

Basically the script polls the Veeam backup server every 10 seconds. When a Veeam backup job has started, the script finds the associated storage.  If the storage is on a VMstore and QoS is set, then the QoS is cleared.  After the job is done, the script restores the QoS settings.  A TGC server is required since the script connects only to a TGC.

Below is a diagram of Veeam_Backup_Tintri_QoS.ps1 (Backup Script) running in the Veeam backup server.


However the backup script could be in a separate VM.

This script could possibly be used as a template for other backup vendors.  Results may very.  Veeam made it straight-forward with a PowerShell plug-in.

Until next time,

– Rick –