SYS-CON MEDIA Authors: Pat Romanski, Gary Arora, Zakia Bouachraoui, Yeshim Deniz, Liz McMillan

Blog Feed Post

iControl 101 - #24 - Folders

Bucket

Way back in time (well, not so way back), configuration objects were stored in one location in the configuration.  For the sake of this article, we’ll call this the root “bucket”.  This worked fine for small organizations but we found that as companies grew and, as a consequence, the number of applications they needed to support increased, it became more difficult to manage all the objects in a single “bucket”.

  • vs_1
  • vs_2
  • pool_1
  • pool_2
  • monitor_1
  • monitor_2

Buckets

In BIG-IP version 9.4, we introduced the concept of “Partitions”.  With Partitions, you could create additional “buckets” of configuration objects.  Each partition could contain objects and have it’s own set of authorization roles protecting them allowing the administrator to group them together and allow each application team to manage those objects without exposing access to objects they weren’t in control of.  I discussed this interface in my article titled: iControl 101 - #08 - Partitions.  A layout of the previously defined objects could now look like the following:

  • /APP1
    • vs_1
    • pool_1
    • monitor_1
  • /APP2
    • vs_2
    • pool_2
    • monitor_2

This still has the limitation in that it’s not easy to determine which groupings of objects (APP1, APP2, …) belong together.  The next logical extension is to allow arbitrary “buckets” of objects which is what I will talk about for the rest of this article.

Buckets of Buckets

BIG-IP version 11, introduced the concepts of Folders.  Here’s an excerpt from the iControl SDK’s reference page for the Folder interface:

A folder stores an arbitrary set of configuration objects. The system uses folders to control access to objects and to control synchronization of these objects within a device group. Folders are organized in a tree heirarchy, much like the folders or directories on a computer's file system. Objects stored in folders are referenced by the name of the individual object, preceded by its folder name, preceded by the names of any parent folders, up to the root folder (/), all separated by slashes (/), e.g., /george/server/virt-a. Note: methods to access the active folder for a session are found in the System::Session interface.

So, now we can have the objects look something like this

  • /APPGROUP
    • /APP1
      • vs_1
      • pool_1
      • monitor_1
    • /APP2
      • vs_2
      • pool_2
      • monitor_2

Since I’m a console-kind-of-guy at heart, and how folders are very similar to directories in a file system, I figured, I’d write a companion sample for this article that emulated a directory shell allowing you to navigate through, create, remove, and modify folders while illustrating how to use the various iControl methods for those tasks.

The Application: A Folder Shell

I wrote this application in PowerShell, but it could have just as easily been coded in Java, Perl, Python, .Net, or whatever you tend to use with your projects.  Let’s take a look at the implementation, the actions that it performs, and some sample output

Initialization

This application will take only three parameters as input.  The address of the BIG-IP,  and the username and password for authentication.  The main application loop consists of:

  1. Verifying the connection information is valid with the code in the Do-Initialize method.
  2. Setting the prompt to the users current folder
  3. Reading a command from the user and passing it to the Process-Input function described below.
param (
  $bigip = $null,
  $uid = $null,
  $pwd = $null
)

Set-PSDebug -strict;

# Global Script variables
$script:DEBUG = $false;
$script:FOLDER = $null;
$script:RECURSE = "STATE_DISABLED";

function Do-Initialize()
{
  if ( (Get-PSSnapin | Where-Object { $_.Name -eq "iControlSnapIn"}) -eq $null )
  {
    Add-PSSnapIn iControlSnapIn
  }
  $success = Initialize-F5.iControl -HostName $bigip -Username $uid -Password $pwd;
  
  return $success;
}

# Main Application Logic
if ( ($bigip -eq $null) -or ($uid -eq $null) -or ($pwd -eq $null) )
{
  usage;
}

if ( Do-Initialize )
{
  $s = Get-RecursiveState;
  while(1)
  {
    $prompt = Get-Prompt;
    # Not using Read-Host here so we can support responses starting with !
    $host.UI.Write($prompt);
    $i = $host.UI.ReadLine().Trim();
    Process-Input $i;
  }
}
else
{
  Write-Error "ERROR: iControl subsystem not initialized"
}

The Main Application Loop

The main application loop passes commands to the Process-Input function.   A wildcard match is performed against the passed in command and if there is a match, control is passed to the appropriate handler function.   I won’t describe them all here but they should be fairly self explanatory.  The main types of actions are described in the upcoming sections.

function Process-Input()
#----------------------------------------------------------------------------
{
  param($i);
  Debug-Message "< $($MyInvocation.MyCommand.Name) '$i' >";
  if ( $i.Length -gt 0 )
  {
    Debug-Message "CommandLine: '$i'...";
    switch -Wildcard ($i.Trim().ToLower())
    {
      ""        { break; }
      "cd *"    { Change-Folder (Get-Args $i); }
      "cd"      { Get-CurrentFolder; }
      "d"       { $script:DEBUG = -not $script:DEBUG; }
      "dir"     { Get-ChildFolders | Sort-Object Folder; }
      "gd *"    { Get-FolderDescription (Get-Args $i); }
      "h"       { Show-Help; }
      "ls"      { Get-ChildFolders | Sort-Object Folder; }
      "ls -l"   { Get-ChildFolders -Long | Sort-Object Folder; }
      "ls -lr"  { Get-ChildFolders -Recurse -Long | Sort-Object Folder; }
      "ls -r"   { Get-ChildFolders -Recurse | Sort-Object Folder; }
      "md *"    { Create-Folder (Get-Args $i); }
      "mkdir *" { Create-Folder (Get-Args $i); }
      "pwd"     { Get-CurrentFolder; }
      "r"       { Set-RecursiveState; }
      "r *"     { Set-RecursiveState (Get-Args $i); }
      "rd *"    { Remove-Folder (Get-Args $i); }
      "rmdir *" { Remove-Folder (Get-Args $i); }
      "sd *"    { Set-FolderDescription (Get-Args $i); }
      "q"       { exit; }
      "! *"     { Execute-Command (Get-Args $i); }
      "$*"      { Execute-Command $i.SubString(1); }
      ".."      { Move-Up; }
      default   { Show-Help; }
    }
  }
}

Querying The Current Folder

The location of the users current, or “active”, folder is determined by calling the System.Session.get_active_folder() method.  This is a server side variable that is stored during the lifetime of the iControl Portals instance and tied to the current authenticated user.  This Get-CurentFolder function does some internal caching with the $script:FOLDER variable to avoid repetitive calls to the server.  The caching can be overridden by passing in the “-Force” argument to the function causing a forced call to query the active folder.  The value is then returned to the calling code.

function Get-CurrentFolder()
{
  param([switch]$Force = $false);
  
  if ( ($($script:FOLDER) -eq $null) -or ($Force) )
  {
    $folder = (Get-F5.iControl).SystemSession.get_active_folder();
  }
  else
  {
    $folder = $script:FOLDER;
  }
  $folder;
}

Listing the Child Folders

The Get-ChildFolders function has several options you can use with it.  If “-Recurse” is passed to the function, the recursive query state is set to STATE_ENABLED telling the server to return all child objects in all child folders relative to the currently active one.  This is similar to a “dir /s” on Windows or “ls -r” on Unix.  The second parameter is “-Long”.  If this is passed in, then a “long” listing will be presented to the user containing the folder name, description, and device group.  Folder descriptions are described below while I’m leaving device groups to be a task for the reader to follow up on.

The function gathers the requested information and then packs the output into objects and returns them along the PowerShell pipeline to the calling code.

function Get-ChildFolders()
{
  param([switch]$Recurse = $false, [switch]$Long = $false);

  if ( $Recurse )
  {
    $oldstate = (Get-F5.iControl).SystemSession.get_recursive_query_state();
    (Get-F5.iControl).SystemSession.set_recursive_query_state("STATE_ENABLED");
  }
  $folders = (Get-F5.iControl).ManagementFolder.get_list();
  if ( $Recurse -and ($oldstate -ne "STATE_ENABLED") )
  {
    (Get-F5.iControl).SystemSession.set_recursive_query_state($oldstate);
  }
  
  $descriptions = (Get-F5.iControl).ManagementFolder.get_description($folders);
  $groups = (Get-F5.iControl).ManagementFolder.get_device_group($folders);
  
  $curfolder = Get-CurrentFolder;
  if ( $curfolder -eq "/" ) { $curfolder = "ZZZZZZZZ"; }
  
  for($i=0;$i-lt$folders.length;$i++)
  {
    if ( $Long )
    {
      $o = 1 | select "Folder", "Description", "DeviceGroup";
      $o.Folder = $folders[$i].Replace($curfolder, "");
      $o.Description = $descriptions[$i];
      $o.DeviceGroup = $groups[$i];
    }
    else
    {
      $o = 1 | select "Folder";
      $o.Folder = $folders[$i].Replace($curfolder, "");
    }
    $o;
  }
}

Changing The Current Folder

The Change-Folder function emulates the “cd” or “chdir” functionality in command shells.  The folder parameter can either be a fully qualified folder name (ie /PARTITION1/FOLDER1/SUBFOLDER), a relative child folder path (ie. FOLDER1/SUBFOLDER, SUBFOLDER, etc), or the special “..” folder which means to go up one level on the folder hierarchy.

function Change-Folder()
#----------------------------------------------------------------------------
{
  param($folder);
  
  Debug-Message "Setting active folder to '$folder'";
  if ( $folder -eq ".." )
  {
    Move-Up;
  }
  else
  {
    (Get-F5.iControl).SystemSession.set_active_folder($folder);
  }
  $f = Get-CurrentFolder -Force;
}

Moving Up A Folder

I wrote a special function to move up one folder from the currently active folder.  The Move-Up function parses the current folder and moves up a path using the PowerShell Split-Path cmdlet.  If the path isn’t the top folder (meaning it has a parent), then the call to System.Session.set_active_folder() is made and the new current folder is queried and cached for future use.

function Move-Up()
{
  $folder = Get-CurrentFolder;
  $parent = (Split-Path $folder).Replace('\', '/');
  if ( $parent.Length -gt 0 )
  {
    Debug-Message "Setting active folder to '$parent'";
    (Get-F5.iControl).SystemSession.set_active_folder($parent);
  }
  $f = Get-CurrentFolder -Force;
}

Creating And Removing Folders

Navigating folders is fun, but it’s more fun to create and destroy them!  The Create-Folder and Remove-Folder functions do just that.  They call the Management.Folder.create() and Management.Folder.delete_folder() methods to do these actions.

function Create-Folder()
{
  param($folder);
  Debug-Message "Creating folder '$folder'";
  (Get-F5.iControl).ManagementFolder.create($folder);
  Write-Host "Folder '$folder' successfully created!";
}

function Remove-Folder()
{
  param($folder);
  Debug-Message "Removing folder '$folder'";
  (Get-F5.iControl).ManagementFolder.delete_folder($folder);
  Write-Host "Folder '$folder' successfully removed!";
}

Folder Descriptions

One great feature of folders is the ability to attach a description to it.  A folder is just another type of object and it’s sometimes useful to be able to store some metadata in there with information that you can’t fit into the name.  The Set-FolderDescription and Get-FolderDescription functions call the Management.Folder.set_description() and Management.Folder.get_description() iControl methods to, well, get and set the descriptions.

function Set-FolderDescription()
{
  param($cmd);
  $tokens = $cmd.Split(" ");
  if ( $tokens.Length -eq 1 )
  {
    $f = $cmd;
    $d = "";
    Debug-Message "Setting folder '$folder' description to '$d'";
    (Get-F5.iControl).ManagementFolder.set_description($f, $d);
    Get-FolderDescription $f;
  }
  elseif ( $tokens.Length -gt 1 )
  {
    # folder description goes here
    $f = $tokens[0];
    $d = $tokens[1];
    for($i=2; $i-lt$tokens.Length; $i++)
    {
      $d += " ";
      $d += $tokens[$i];
    }
    Debug-Message "Setting folder '$f' description to '$d'";
    (Get-F5.iControl).ManagementFolder.set_description($f, $d);
    Get-FolderDescription $f;
  }
  else
  {
    Show-Help;
  }
}

function Get-FolderDescription()
{
  param($folder);
  Debug-Message "Retrieving folder description for '$folder'";
  $descriptions = (Get-F5.iControl).ManagementFolder.get_description($folder);
  $descriptions[0];
}

Controling The Recursive State For Queries

I touched at the recursive query option above when I was showing how to query child folders.  This recursive state not only applies to folder queries, but all queries across the iControl API!  The Set-RecursiveState function sets this configuration variable to either STATE_ENABLED or STATE_DISABLED.  If the recursive_query_state is set to STATE_DISABLED, then queries will only return objects in the current active folder.  But, if it’s set to STATE_ENABLED, then it will return all child objects in all child folders.  So, by setting this one value, you can determine whether you get just the objects (pools, virtuals, vlans, monitors, etc) in the current folder, or all of them in all child folders.  Cool stuff!

function Set-RecursiveState()
{
  param($state = $null);
  
  $newState = "STATE_DISABLED";
  if ( $state -eq $null )
  {
    # toggle
    $oldState = (Get-F5.iControl).SystemSession.get_recursive_query_state();
    if ( $oldState -eq "STATE_DISABLED" )
    {
      $newState = "STATE_ENABLED";
    }
  }
  else
  {
    # set
    if ( $state.ToLower().Contains("enable") )
    {
      $newState = "STATE_ENABLED";
    }
  }
  $script:RECURSE = $newState;
  (Get-F5.iControl).SystemSession.set_recursive_query_state($newState);
  Write-Host "Recursive State set to '$newState'";
}

function Get-RecursiveState()
{
    $oldState = (Get-F5.iControl).SystemSession.get_recursive_query_state();
    $script:RECURSE = $oldState;
    $oldState;
}

Executing Arbitrary iControl Commands

I threw this in to help illustrate how the recursive state described above works.  By passing in the command “! LocalLBVirtualServer.get_list()”, that iControl call will be executed and the results passed to the output.  By using the Invoke-Expression cmdlet, any iControl call can arbitrarily be made within the shell.  Again, cool stuff!

function Execute-Command()
{
  param($cmd);
  
  $fullcmd = $cmd;
  if ( -not $fullcmd.ToLower().StartsWith("(get-f5.icontrol).") )
  {
    $fullcmd = "(Get-F5.iControl).$cmd";
  }
  
  Debug-Message "Executing command '$fullcmd'";
  Invoke-Expression $fullcmd;
}

A Demo Walkthrough

Conclusion

Folders will allow administrators more control of grouping their objects together, thus enabling them with the long-term manageability of those objects.  This example illustrated how to use the iControl methods to interact with folders and, hopefully, in doing so, showed the ease at building powerful iControl solutions.

The Source Code

The Full Source can be found in the iControl Wiki in the CodeShare entry titled PowerShellManagementFolder

Related Content on DevCentral

 

Read the original blog entry...

More Stories By Joe Pruitt

Joe Pruitt is a Principal Strategic Architect at F5 Networks working with Network and Software Architects to allow them to build network intelligence into their applications.

Latest Stories
While a hybrid cloud can ease that transition, designing and deploy that hybrid cloud still offers challenges for organizations concerned about lack of available cloud skillsets within their organization. Managed service providers offer a unique opportunity to fill those gaps and get organizations of all sizes on a hybrid cloud that meets their comfort level, while delivering enhanced benefits for cost, efficiency, agility, mobility, and elasticity.
Isomorphic Software is the global leader in high-end, web-based business applications. We develop, market, and support the SmartClient & Smart GWT HTML5/Ajax platform, combining the productivity and performance of traditional desktop software with the simplicity and reach of the open web. With staff in 10 timezones, Isomorphic provides a global network of services related to our technology, with offerings ranging from turnkey application development to SLA-backed enterprise support. Leadin...
DevOps has long focused on reinventing the SDLC (e.g. with CI/CD, ARA, pipeline automation etc.), while reinvention of IT Ops has lagged. However, new approaches like Site Reliability Engineering, Observability, Containerization, Operations Analytics, and ML/AI are driving a resurgence of IT Ops. In this session our expert panel will focus on how these new ideas are [putting the Ops back in DevOps orbringing modern IT Ops to DevOps].
Darktrace is the world's leading AI company for cyber security. Created by mathematicians from the University of Cambridge, Darktrace's Enterprise Immune System is the first non-consumer application of machine learning to work at scale, across all network types, from physical, virtualized, and cloud, through to IoT and industrial control systems. Installed as a self-configuring cyber defense platform, Darktrace continuously learns what is ‘normal' for all devices and users, updating its understa...
Enterprises are striving to become digital businesses for differentiated innovation and customer-centricity. Traditionally, they focused on digitizing processes and paper workflow. To be a disruptor and compete against new players, they need to gain insight into business data and innovate at scale. Cloud and cognitive technologies can help them leverage hidden data in SAP/ERP systems to fuel their businesses to accelerate digital transformation success.
Concerns about security, downtime and latency, budgets, and general unfamiliarity with cloud technologies continue to create hesitation for many organizations that truly need to be developing a cloud strategy. Hybrid cloud solutions are helping to elevate those concerns by enabling the combination or orchestration of two or more platforms, including on-premise infrastructure, private clouds and/or third-party, public cloud services. This gives organizations more comfort to begin their digital tr...
Most organizations are awash today in data and IT systems, yet they're still struggling mightily to use these invaluable assets to meet the rising demand for new digital solutions and customer experiences that drive innovation and growth. What's lacking are potent and effective ways to rapidly combine together on-premises IT and the numerous commercial clouds that the average organization has in place today into effective new business solutions.
Keeping an application running at scale can be a daunting task. When do you need to add more capacity? Larger databases? Additional servers? These questions get harder as the complexity of your application grows. Microservice based architectures and cloud-based dynamic infrastructures are technologies that help you keep your application running with high availability, even during times of extreme scaling. But real cloud success, at scale, requires much more than a basic lift-and-shift migrati...
David Friend is the co-founder and CEO of Wasabi, the hot cloud storage company that delivers fast, low-cost, and reliable cloud storage. Prior to Wasabi, David co-founded Carbonite, one of the world's leading cloud backup companies. A successful tech entrepreneur for more than 30 years, David got his start at ARP Instruments, a manufacturer of synthesizers for rock bands, where he worked with leading musicians of the day like Stevie Wonder, Pete Townsend of The Who, and Led Zeppelin. David has ...
Darktrace is the world's leading AI company for cyber security. Created by mathematicians from the University of Cambridge, Darktrace's Enterprise Immune System is the first non-consumer application of machine learning to work at scale, across all network types, from physical, virtualized, and cloud, through to IoT and industrial control systems. Installed as a self-configuring cyber defense platform, Darktrace continuously learns what is ‘normal' for all devices and users, updating its understa...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Addteq is a leader in providing business solutions to Enterprise clients. Addteq has been in the business for more than 10 years. Through the use of DevOps automation, Addteq strives on creating innovative solutions to solve business processes. Clients depend on Addteq to modernize the software delivery process by providing Atlassian solutions, create custom add-ons, conduct training, offer hosting, perform DevOps services, and provide overall support services.
Contino is a global technical consultancy that helps highly-regulated enterprises transform faster, modernizing their way of working through DevOps and cloud computing. They focus on building capability and assisting our clients to in-source strategic technology capability so they get to market quickly and build their own innovation engine.
When applications are hosted on servers, they produce immense quantities of logging data. Quality engineers should verify that apps are producing log data that is existent, correct, consumable, and complete. Otherwise, apps in production are not easily monitored, have issues that are difficult to detect, and cannot be corrected quickly. Tom Chavez presents the four steps that quality engineers should include in every test plan for apps that produce log output or other machine data. Learn the ste...
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...