The Men & Mice Blog

Men & Mice Suite Version 7.3 – Plugging into VMware while having a REST

Posted by Men & Mice on 11/10/16 9:12 AM

Men & Mice Suite Version 7.3 has arrived - and not a minute too soon! Yet considering that it’s jam-packed with goodies such as a brand-new REST API, VMware vRealize Orchestrator plug-in and further support for Windows Server 2016, it was definitely worth the wait.

Let’s take a quick peek at what Version 7.3 has in store for our customers.

Taking a break with the REST API

API.png

API. In the world of the Internet, it means Application Programming Interface. In the world of the Icelandic language (where Men & Mice has its roots) it means … monkey. Literally. And maybe just as well – a good API, with or without hair, really does seem to make life so much better.

Monkey business or no monkey business, the Men & Mice REST API is sure to offer customers a very welcome extra set of hands - and feet, so to speak – with which to create workflows and write handy scripts for the import and export of data, amongst other things.

Used by browsers, REST (Representational State Transfer) is often considered to be the language of the internet. By using HTTP requests to GET, POST, PUT and DELETE data, REST paves the way for two computers to communicate over the internet by one acting as a web server and the other as a web browser. Making use of a stateless protocol, RESTful services exhibit particularly fast performance, reliability and scalability.

The Men & Mice REST API includes all the functionality of the existing Men & Mice SOAP API and JSON-RPC, but delivers the added advantages of ease of use, combined with a rich set of tools and support libraries. Additionally REST, as a resource-based instead of a standards-based API, means users gain considerably greater operational flexibility.

More information on how to get the most out of REST can be found in the Men & Mice REST API article.

Plugging in where it matters – VMware vRealize Orchestrator Plug-In

Men & Mice takes a further step towards simplifying virtualization by introducing the VMware vRealize Orchestrator plug-in. Designed to integrate seamlessly within the VMware Orchestrator framework, the Men & Mice Suite VMware plug-in allows for fast and efficient provisioning of virtual machines.

 

vmware_plugin.png

 

When a Men & Mice Suite user puts in a request for a new virtual machine (VM), the vRealize Orchestrator receives the next available IP address from the requested subnet, together with other essential configuration information. vCenter creates the VM and communicates the changes back to the Men & Mice Suite, which then updates DNS infrastructure accordingly. Additionally:

  • the Men & Mice Suite’s custom properties allow further customization of the VM’s visibility and status.
  • VM information retained in the Men & Mice Suite enables VM tracking, synchronization and updates, including the release of IP addresses after a virtual server is taken down.
  • the Men & Mice Suite talks to DNS servers and registers DNS entries and other changes, such as updates to DNS policies, thereby consolidating DNS data required by the vRealize Orchestrator.

By plugging into the vRealize Orchestrator, the Men & Mice Suite enables integrated functionality that not only saves time, but also strengthens security, eliminates errors of configuration and ensures improved and continuously synchronized network manageability.

Windows Server 2016 Support Released in Tandem with General Availability

Men & Mice Suite support for primary Windows Server 2016 DNS and DHCP features was already included in Version 7.2, released in May 2016. A stand-out feature was support for Response Rate Limiting, which significantly reduces the impact of a Denial of Service (DoS) attack on servers.

With Windows Server 2016 achieving General Availability in September 2016, Men & Mice expands its support for the following additional Windows Server 2016 features:

DNS policies

DNS policies grant a user control over how queries are handled based on specific criteria. These criteria can, for example, be used in the following scenarios:

  • High availability of DNS services
  • Traffic management
  • Split brain DNS 
  • Filtering
  • Forensics
  • Redirection based on date/time

Specific types of policies are:

  • Zone transfer policies
    Essentially used to define how zone transfers take place, zone transfer policies control zone transfer permission on the server level or the zone level. 
  • Recursion policies
    Control how the DNS server performs recursion for a query. 
  • DNS query resolution policies
    Used to specify how incoming DNS queries are handled by the DNS server. 

IPv6 root hints

The IPv6 root servers can now be used for performing name resolution. 

DANE TLSA records

DANE, or DNS-based Authentication of Named Entities, allows a domain owner to specify in a particular DNS record which certificates authorities are allowed to issue for the domain.

The Men & Mice Suite Release Notes provide more detail on other minor improvements and fixes that form part of the Version 7.3 Release.

That wraps it up for a quick round-up of all things new and shiny that the Men & Mice Suite Version 7.3 has to offer. If you’d like to jump right in and try out these new features, treat yourself to a Version 7.3 free trial. 

Men & Mice Suite trial

 

Coming up in December is the last in our 2016 series of webinars, this time focusing on DNS high availability tools. Don’t forget to sign up!

 

Topics: Men & Mice Suite, DDI, API, VMware

Men & Mice Web Service: REST API

Posted by Men & Mice on 11/10/16 6:42 AM

Introduction

Men & Mice is expanding our web service offerings by adding a REST API web service to the existing SOAP/XML and JSON-RPC services.

This article serves as an introduction to the Men & Mice REST API, providing information on background, purpose and functionality.

What is REST?

REST, or more specifically Representational State Transfer, is often described as the language of the Internet. An architectural style for distributed hypermedia systems, REST was first introduced by Roy Fielding in his doctoral dissertation at UC Irvine in the year 2000.

Fielding’s experience as one of the principal authors of the HTTP specification led to his development of REST as a set of principles and constraints for communication between computers on the Internet. The six architectural constraints unique to REST are client-server separation, statelessness, cacheability, uniform interface, layered systems and code on demand – the latter being the only constraint that is optional.

According to Fielding, the purpose of creating REST was to simplify and enhance the distribution of data between systems. Given how widespread REST has become, it’s safe to say Fielding’s mission has been accomplished. Architectural properties affected by REST are performance, scalability, simplicity of a uniform interface, modifiability of components, visibility of communication between components, portability of components and reliability.

Since its introduction, REST has gained much popularity, likely due to its positive effect on architectural properties and its simplicity, both particularly critical in the era of exponential increases in cloud usage offerings. Today, the majority of new web services are designed as RESTful[1] services instead of SOAP/XML, JSON-RPC, or other types of communications.


Why add a REST API to the Men & Mice Suite?

Men & Mice web services in the form of SOAP/XML and JSON-RPC provide an extensive set of commands to configure and control all aspects of the Men & Mice Suite. However, REST has become the first choice of communication for web service applications. By making use of a stateless protocol, RESTful services exhibit particularly fast performance, reliability and scalability. Additionally, REST’s simplicity generally makes it easier for users to get started and engage with the service.

The greatest difference for users between SOAP and REST is that SOAP as a standards-based web service access protocol is more rigid in execution, whereas REST as a resource-based web service provides greater flexibility. In most cases, a user will only need a browser or a simple command line tool such as cURL to access data from a RESTful web service.


How does the Men & Mice REST API work?                                               

In REST, the focus is on resources. You specify a resource with a URL (Uniform Resource Location) and then apply an operation on the resource using an HTTP method.

The Men & Mice REST API supports the four most common HTTP methods: GET, PUT, POST and DELETE.

  • GET – Retrieve a resource (read)
  • PUT - Modify an existing resource (update)
  • POST - Add a new resource (create)
  • DELETE - Remove a resource (delete)

The resources or the objects found in the Men & Mice Suite are:

  • AddressSpaces
  • ADForests
  • ADSiteLinks
  • ADSites
  • ChangeRequests
  • CloudNetworks
  • Clouds
  • Devices
  • DHCPAddressPools
  • DHCPExclusions
  • DHCPGroups
  • DHCPReservations
  • DHCPScopes
  • DHCPServers
  • DNSRecords
  • DNSServers
  • DNSViews
  • DNSZones
  • Folders
  • Groups
  • Interfaces
  • IPAMRecords
  • Ranges
  • Roles
  • Users

 
An example of a URL referring to a DNS zone would be:

            http://mmsuite.company.com/mmws/api/DNSZones


To get all the zones defined in the Men & Mice Suite you would use HTTP GET:

            GET http://mmsuite.company.com/mmws/api/DNSZones


To get a specified zone, e.g. test.menandmice.com, you would also use HTTP GET, but with a reference to the specific zone:

            GET http://mmsuite.company.com/mmws/api/DNSZones/test.menandmice.com.

 
The Men & Mice REST API understands two types of content: JSON and XML. The content type of the response will depend on the type of content in the request. If there is no content in the body of the request, the web service will check for a clue in the HTTP header fields "Content-Type" and "Accept". If either of the fields exist and contain "application/json", it will return a JSON formatted response. If either of the fields exist and contain "application/xml" or "application/soap+xml", it will return an XML formatted response. If no clues can be found, it will select JSON as the response format.

It’s also possible to mix these two content types during the same session and the web service will simply respond with JSON if this was a JSON request, or XML if the content type detected was XML.

The Men & Mice REST API is built on the same code base as both SOAP/XML and JSON-RPC. However, some of the commands you will see in SOAP/XML and JSON-RPC are not a part of the REST API. The reason for this is that in REST, the focus is on resources, not commands. You can, however, execute all the commands found in SOAP/XML and JSON-RPC using the URL: api/command/<command>.  For example, to get all orphan DNS records found in your Men & Mice Suite, you can say: 

            GET http://mmsuite.company.com/mmws/api/command/GetOrphanReverseDNSRecords


Orphan DNS records are PTR records where the corresponding A/AAAA record is missing.

For possible commands, please refer to the Men & Mice SOAP reference manual.


Arguments

The Men & Mice REST API supports many arguments that can be added to the URL. For example, when getting zones you can say:

            GET http://mmsuite.company.com/mmws/api/DNSZones?limit=2&pretty=true

This would return to you a list of zones in the Men & Mice Suite. At the most two lists would be returned, with the output made easier to read by adding lines and spaces.

There are a few arguments that are always available, no matter what resource you are referring to:

  • pretty – If set to ‘true’ it will make the response more readable.
  • server - Name or address of a Men & Mice Central server to connect to.
  • loginName - The name of the user who wants to log in.
  • password - The password for the user.
  • session - An ID of a valid session.

These arguments are only optional. You don't need to use arguments to log on to a server. The Men & Mice REST API offers different types of authentication, such as Basic Authentication, Windows NTLM and Kerberos. Note that in order for a user to be able to use the REST web service, the user has to have the applicable permission to use the web user interface.

There are some arguments that you can provide in many cases when using the HTTP GET method:

  • filter - Filtering criteria for the result returned.
  • offset - Specifies the offset to use when listing the results. A value of 0 starts with the first result.
  • limit - The maximum number of results to return.
  • sortBy - The name of the field to sort by.
  • sortOrder - The sort order to use.

When adding, or changing a resource, you will need to provide some data. In most cases the data will be provided as a body of the HTTP request. Data can also be provided as an argument. The server will understand that you are providing something that should be a part of the data. For example, when adding a DNS record, instead of providing a body with the HTTP method POST, you can say instead:

            POST http://mmsuite.company.com/mmws/api/DNSZones/test.menandmice.com./

                        DNSRecords?dnsRecord={ "name": "restest", "type": "A", "data": "1.2.3.11"}

 

Filter arguments

The filter argument is a powerful argument that can be provided with most of the HTTP GET methods. It allows you to limit the result you get to only the items you want to see. You can use wildcards and regular expressions with a filter.

The filter has the following format:

            [<property>:][!][^]<value>[$]

Both <property> and <value> can be quoted using " in case they contain characters that might confuse the filter, such as a space. The "!", "^" and "$" are all optional where the "!" symbol means ‘does-not-match’, the "^" sign means ‘starts-with’, and the "$" sign means ‘ends-with’.

 

Some examples:

Filtering by string "mycorp" on all properties:

            filter=mycorp

Filtering by string "mycorp" on property "name":

            filter=name:mycorp

Filtering by string starting with "mycorp" on property "name":

            filter=name:^mycorp

Filtering by string not ending with "mycorp" on property "name":

            filter=name:!mycorp$


If the text must contain whitespace, it must be quoted to filter by string, not starting with "requested by" on property "comment":

            filter=comment:"!^requested by"


The same applies if the property name contains whitespace, so to filter by string "mycorp" on property "Company Name":

            filter="Company Name":mycorp


When getting zones, the following filter can be used to get all master zones, excluding reverse zones:

            filter=type:^Master$ name:!arpa.$


Getting only zones with name "domain.com.":

            filter=name:^domain.com.$


Getting only zones ending with the name "domain.com.":

            filter=name:domain.com.$


The following examples illustrate the usage of a filter when getting DNSRecords. The next filter will get all records containing the name "time" from within the specified zone:

            filter=name:^time$

Getting all A records from within the specified zone:

            filter=type:^A$

Getting all records with data "ntp":

            filter=data:^ntp$

Getting all A records with the name "time":

            filter=type:^A$ name:^time$

 

Men & Mice REST API in action - examples 

Several great tools are available for working with web services, such as Postman and cURL.

Postman is highly recommended, especially for those interested in testing web services. Postman allows the testing of requests, after which it can be asked to generate code snippets for that request in different programming languages.

cURL is a popular command line tool that is available on most platforms. It is installed by default on most Unix flavors. For Windows, it can be downloaded from https://curl.haxx.se  

cURLl can be handy when you want to export data or combine data with simple scripts. The examples on the following pages were created by trying out the REST API using cURL.

For these examples, let's assume that our web service is running on the server mmsuite.company.com, our user name is "john" and our password is "secret". 

$ curl --user john:secret -X GET http://mmsuite.company.com/mmws/api/DNSServers

{

  "result": {

    "dnsServers": [

      {

        "ref": "DNSServers/3",

        "name": "a-win2008r2.mmsuite.company.com.",

        "resolvedAddress": "172.17.0.17",

        "port": 1337,

        "type": "MS",

        "state": "OK",

        "customProperties": {},

        "subtype": "Win2008",

        "enabled": true

      }

    ],

    "totalResults": 1

  }

}


Since we didn’t provide any information about what kind of content type we wanted, the server responded with JSON output. If we want to get the result back in XML format, we can simply add the XML "Content-Type".

$ curl --user john:secret --header "Content-Type: application/xml" -X GET http://mmsuite.company.com/mmws/api/DNSServers

<response>

  <result>

    <dnsServers>

      <dnsServer>

        <ref>DNSServers/3</ref>

        <name>a-win2008r2.mmsuite.company.com.</name>

        <resolvedAddress>172.17.0.17</resolvedAddress>

        <port>1337</port>

        <type>MS</type>

        <state>OK</state>

        <customProperties />

        <subtype>Win2008</subtype>

        <enabled>1</enabled>

      </dnsServer>

    </dnsServers>

    <totalResults>1</totalResults>

  </result>

</response>

 

 
Now let's try to use filters and get a list of all reverse zones in the Men & Mice Suite.

$ curl --user john:secret -X GET "http://mmsuite.company.com/mmws/api/ DNSZones?filter=name:in-addr.arpa.&pretty=true"

{

  "result": {

    "dnsZones": [

      {

        "ref": "DNSZones/10",

        "name": "1.5.2.in-addr.arpa.",

        "dynamic": false,

        "adIntegrated": false,

        "dnsViewRef": "DNSViews/3",

        "authority": "a-win2008r2.remote.mm.lab.",

        "type": "Slave",

        "dnssecSigned": false,

        "kskIDs": "",

        "zskIDs": "",

        "customProperties": {}

      },

      {

        "ref": "DNSZones/11",

        "name": "10.in-addr.arpa.",

        "dynamic": false,

        "adIntegrated": false,

        "dnsViewRef": "DNSViews/3",

        "authority": "a-win2008r2.remote.mm.lab.",

        "type": "Slave",

        "dnssecSigned": false,

        "kskIDs": "",

        "zskIDs": "",

        "customProperties": {}

      }

    ],

    "totalResults": 2

  }

}


Notice the quotation marks around the URL and the arguments. The reason for this is that we are using characters such as "&" that might confuse the command line. By putting quotation marks around it, we are saying that everything inside the quote is a part of the data and should not be interpreted in a different way.

Here is another great example of how powerful the filters are. Let's find all A records starting with “vm” in the zone dev.lab.

$ curl --user john:secret -X GET "http://mmsuite.company.com/mmws/api/ DNSZones/dev.lab./DNSRecords?filter=type:^A$ name:^vm&pretty=true"

{

  "result": {

    "dnsRecords": [

      {

        "ref": "DNSRecords/374",

        "name": "vm-1",

        "type": "A",

        "ttl": "",

        "data": "10.4.4.3",

        "comment": "",

        "enabled": true,

        "dnsZoneRef": "DNSZones/20"

      },

      {

        "ref": "DNSRecords/376",

        "name": "vm-1",

        "type": "A",

        "ttl": "",

        "data": "10.4.4.1",

        "comment": "",

        "enabled": true,

        "dnsZoneRef": "DNSZones/20"

      },

      {

        "ref": "DNSRecords/377",

        "name": "vm-1",

        "type": "A",

        "ttl": "",

        "data": "10.4.4.2",

        "comment": "",

        "enabled": true,

        "dnsZoneRef": "DNSZones/20"

      }

    ],

    "totalResults": 3

  }

}


Note that all of the GET commands can be executed in a simple browser. When you enter a URL in a browser, it will send an HTTP GET command to the server you are referring to. This can become handy if you don't have cURL installed. You can try this by opening a browser, entering the address of your Men & Mice Web Server and appropriating a REST resource, e.g.

            http://menandmice.com mmws/mmws/api/DNSZones&pretty=true

If your Men & Mice web service is configured to allow Basic Authentication or Windows Authentication (NTLM or Kerberos), it will prompt you for a user name and password.


Scripts or programming languages

Now what about scripts or programming languages? Because, as mentioned earlier, REST is the simplest and most popular choice when creating a web service, it is usually well supported in all languages. As an example, let's look at how we would list out all records in a reverse zone that are suspicious. A reverse zone usually only contains NS and PTR records. Other record types are allowed, but usually don't appear there.

We wrote this script using Python. Python is a great scripting language and well-suited to smaller scripts, especially when dealing with JSON and strings. It is a pretty comprehensive language, yet there are plenty of additional libraries to be explored, if one should need something more.

From Python 3 onwards you can use the http.client library to create a REST request. Bear in mind that Python 2.7, however, does not include the http.client library. Since we will be using Python 2.7, we will be using the requests library which is not a part of the standard installation. For information on how to install the requests library, see http://docs.python-requests.org/en/latest/

#!/usr/bin/env python

#

# restDemo.py - list all suspicious records found in

# reverse zones

import requests

 

username = 'john'

password = 'secret'

headers = {'content-type':'application/json'}

url = 'http://mmsuite.company.com/mmws/api/'

params= {'filter' : 'name:in-addr.arpa.'}

 

sess = requests.Session()

resp = sess.get(url + 'DNSZones', params=params, auth=(username, password), headers=headers)

# resp should now contain a list of all reverse zone

if resp.ok:

    for zone in resp.json()['result']['dnsZones']:

        print 'checking zone: ' + zone['name']

        # for each zone get all the records

        resp = sess.get(url + zone['ref'] + '/DNSRecords'

                    , auth=(username, password), headers=headers)

        if resp.ok:

            for rec in resp.json()['result']['dnsRecords']:

                if rec['type'] not in ['SOA', 'NS', 'PTR']:

                    print '\t!!!\t' + rec['name'] + '\t' +

                          rec['type'] + '\t' + rec['data']

 


The output resulting from the script is illustrated in the next box.
 

$ ./restDemo.py

checking zone: 1.5.2.in-addr.arpa.

checking zone: 10.in-addr.arpa.

checking zone: 137.168.192.in-addr.arpa.

checking zone: 4.6.2.in-addr.arpa.

checking zone: 49.10.in-addr.arpa.

checking zone: 7.3.2.in-addr.arpa.

checking zone: 2.2.63.in-addr.arpa.

      !!!   jonas TXT   test

 

 

The script found 7 reverse zones. One of them contained a TXT record which we consider suspicious.

And just for fun, because we love those filters, we could have used them to retrieve only the records that are not of the type SOA, NS and PTR. So the last part of our Python script could have been written like this:

...

        print 'checking zone: ' + zone['name']

 

        params= {'filter' : 'type:!^(SOA or NS or PTR)$'}

        # for each zone get all the records, only get records of

        # all types other then SOA, NS or PTR

        resp = sess.get(url + zone['ref'] + '/DNSRecords', params=params

                    , auth=(username, password), headers=headers)

        if resp.ok:

            for rec in resp.json()['result']['dnsRecords']:

                print '\t!!!\t' + rec['name'] + '\t' +

                       rec['type'] + '\t' + rec['data']

 

Writing the script this way leads to less traffic and less load, which can make a difference if the reverse zones are large.

When creating scripts, it is best to create a new, dedicated user account that only has access to the objects the script needs to function. If you are running Microsoft Windows in an AD environment, the web service can also be configured to allow single sign-on.

Men & Mice REST API: Summary

The primary focus of the Men & Mice development team has always been to simplify the complex task of administering a DDI[2] environment, while retaining network flexibility and maintaining speed, as different network environments and different types of users have very different needs. To achieve this flexibility, our mission has been, and continues to be, developing and providing a rich set of commands, combined with easy user access through a web service interface.

Extending the Men & Mice Suite to include a REST API creates a powerful additional tool for users. With the Men & Mice REST API, users gain easier access to data in the Men & Mice Suite, as well as the means to process it according to their particular needs. The Men & Mice REST API therefore extends the range of tools with which customers can create workflows, write handy scripts to import and export data or generally just develop customized ways to lighten the load of administering the often complex, but vital, daily tasks existing in our technical lives.

[1] REST or RESTful?

REST is the set of architectural constraints and is not dependent on any protocol. Web service APIs are typically called RESTful when they adhere to the REST architectural constraints. Practically every RESTful service uses HTTP as its underlying protocol.

[2] DNS, DHCP and IP Address Management

Topics: Men & Mice Suite, API