SYS-CON MEDIA Authors: Zakia Bouachraoui, Liz McMillan, Yeshim Deniz, Elizabeth White, William Schmarzo

Blog Feed Post

HTTP Request Cloning via iRules, Part 1

One of the requests that I've seen several times over the years is the ability to completely clone web requests across multiple servers. The idea is that you can take the HTTP traffic coming in bound for pool member A and distribute it, in its entirety, to pool member B. Or perhaps members B-G..whatever your needs are. This can be helpful for many reasons, security auditing, test or dev harnesses, archival, etc. Whatever the reasons, this has been a repeated question in the forums and in the field. While clone pool functionality works to some degree for this, it doesn't work quite as desired, and doesn't easily distribute to multiple additional members.

iRules, however, offers a solution.

Using the HSL feature in iRules that, if you remember, allows you to specify a protocol and destination, which can be a pool, we are able to treat this much like sideband connections in v11. By establishing a new connection and sending across the HTTP info as needed we're able to clone the HTTP traffic in its entirety. Let's take a look at how this starts:

   1: when CLIENT_ACCEPTED {
   2:     # Open a new HSL connection if one is not available
   3:     set hsl [HSL::open -proto TCP -pool http_clone_pool]
   4:     log local0. "[IP::client_addr]:[TCP::client_port]: New hsl: $hsl"
   5: }

As you can see, it's straight-forward enough. Using the HSL::open command we set the protocol to TCP and the pool to whichever pool you'd like to clone your HTTP traffic to. Now that we know where and how we're sending the data, we need to figure out which data to send. The only trick with HTTP in this step is that GET and POST requests need to be handled differently. With a POST we will need to collect the data that is being posted so that we can replay it back to the new destination. With a GET we simply forward through the headers of the request. Fortunately determining which is which is a cake walk in iRules, so it's just the collecting and forwarding bit we really need to worry about. This is the real "meat" of this iRule, and even that isn't difficult, it looks like:

   1: when HTTP_REQUEST {
   2:     # Insert an XFF header if one is not inserted already
   3:     # So the client IP can be tracked for the duplicated traffic
   4:     HTTP::header insert X-Forwarded-For [IP::client_addr]
   6:     # Check for POST requests
   7:     if {[HTTP::method] eq "POST"}{
   9:         # Check for Content-Length between 1b and 1Mb
  10:         if { [HTTP::header Content-Length] >= 1 && [HTTP::header Content-Length] < 1048576 }{
  11:             HTTP::collect [HTTP::header Content-Length]
  12:         } elseif {[HTTP::header Content-Length] == 0}{
  13:             # POST with 0 content-length, so just send the headers
  14:             HSL::send $hsl [HTTP::request]
  15:             log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  16:         }
  17:     } else {
  18:         # Request with no payload, so send just the HTTP headers to the clone pool
  19:         HSL::send $hsl [HTTP::request]
  20:         log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  21:     }
  22: }

As you can see this is pretty standard iRules fare for the most part. HTTP::method, HTTP::header, HTTP::collect. Nothing shocking for the most part. The real trick is in the HSL::send command. Note that it's going to "$hs1"? That's the connection we established earlier with the HSL::open command. Now that we have that handle available we're able to easily forward through other traffic. So as you can see in the POSTs with content attached  we're collecting, and anything else we're forwarding along the headers alone. Note that nothing has been sent for the POSTs that have content attached, we've just entered a collect state so the client will continue sending data and we'll store it. That data is then available in the HTTP_REQUEST_DATA event, and we can forward it along when that occurs.  So for those particular requests an additional event will fire:

   1: when HTTP_REQUEST_DATA {
   2:     # The parser does not allow HTTP::request in this event, but it works
   3:     set request_cmd "HTTP::request"
   4:     log local0. "[IP::client_addr]:[TCP::client_port]: Collected [HTTP::payload length] bytes,\
   5:         sending [expr {]string length [eval $request_cmd][ + ]HTTP::payload length[}] bytes total"
   6:     HSL::send $hsl "[eval $request_cmd][HTTP::payload]"
   7: }

Now that the HTTP_REQUEST_DATA event has fired we know our collect has picked up the data we want it to. This event will only fire after a successful HTTP::collect. Once this happens we're ready to forward along the POST and the accompanying data.  After a little expr trickery to convince the parser to allow the HTTP::request command within the HTTP_REQUEST_DATA event (it doesn't think it should work, but it we trick it) we're able to send along the original request and payload data without much hassle. Again making use of the HSL::send command and the $hs1 variable we set up at the beginning makes this process easy.

At this point you now have a functioning iRule that will clone traffic inbound for your Virtual to another pool of your choosing. At this point you are probably asking yourself three questions.

1) Why hasn't this been written before?

2) Where is the version that allows forwarding to multiple other pools?!

3) Why HSL and not sideband connections?

Well, those answers are simple:

1) Because our good friend Hoolio hadn't written it yet! Aaron whipped this together and posted it. I got his okay to write it up and get it out there, so here it is. Keep in mind that this is VERY early in the testing stages and is prone to change/update. I'm sharing it here because I think it's awesome, and don't want it to slip off into the night without being called out. But this is very much a use at your own risk sort of thing for now. I'll update with notes when more testing has been done. Also worth note is that this requires at least version 10.1 or newer to function.

2) It's coming, don't fret. That will be Part 2! You did notice the Part1 in the title didn't you? We can't give it all away at once. Besides that part is still under testing. Releasing it before it's ready wouldn't be prudent. Stay tuned, it's coming.

3) I asked Aaron the exact same thing and here's what he said:

HSL automatically ACKs the server responses, but ignores the data.  From limited testing of both HSL and sideband connections, HSL is also a lot more efficient in handling high connection rates.  Also, HSL is available from 10.1 and sideband only on 11.x.

So there you have it. Sideband connections would work just fine, but HSL allows for a wider audience (10.1 and above), and offers a little added efficiency/ease of use in this particular case. Keep in mind that HSL won't handle many of the more complex scenarios that sideband connections will, hence the tradeoff, but in this particular case HSL seems to win out.

Keep an eye out for the next installment of this two-parter, wherein you'll see how to extend this model to work with multiple clone destinations. For now, any questions/comments are welcome. If you get this up and running in test/dev, please let us know as we'd love any feedback resulting from that testing. Many thanks to hoolio for his continued, outstanding contributions. With that, I'll leave you with the full version of the iRule:

   1: when CLIENT_ACCEPTED {
   2:     # Open a new HSL connection if one is not available
   3:     set hsl [HSL::open -proto TCP -pool http_clone_pool]
   4:     log local0. "[IP::client_addr]:[TCP::client_port]: New hsl: $hsl"
   5: }
   6: when HTTP_REQUEST {
   8:     # Insert an XFF header if one is not inserted already
   9:     # So the client IP can be tracked for the duplicated traffic
  10:     HTTP::header insert X-Forwarded-For [IP::client_addr]
  12:     # Check for POST requests
  13:     if {[HTTP::method] eq "POST"}{
  15:         # Check for Content-Length between 1b and 1Mb
  16:         if { [HTTP::header Content-Length] >= 1 && [HTTP::header Content-Length] < 1048576 }{
  17:             HTTP::collect [HTTP::header Content-Length]
  18:         } elseif {[HTTP::header Content-Length] == 0}{
  19:             # POST with 0 content-length, so just send the headers
  20:             HSL::send $hsl [HTTP::request]
  21:             log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  22:         }
  23:     } else {
  24:         # Request with no payload, so send just the HTTP headers to the clone pool
  25:         HSL::send $hsl [HTTP::request]
  26:         log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  27:     }
  28: }
  29: when HTTP_REQUEST_DATA {
  30:     # The parser does not allow HTTP::request in this event, but it works
  31:     set request_cmd "HTTP::request"
  32:     log local0. "[IP::client_addr]:[TCP::client_port]: Collected [HTTP::payload length] bytes,\
  33:         sending [expr {]string length [eval $request_cmd][ + ]HTTP::payload length[}] bytes total"
  34:     HSL::send $hsl "[eval $request_cmd][HTTP::payload]"
  35: }

Read the original blog entry...

More Stories By Colin Walker

Coming from a *Nix Software Engineering background, Colin is no stranger to long hours of coding, testing and deployment. His personal experiences such as on-stage performance and the like have helped to foster the evangelist in him. These days he splits his time between coding, technical writing and evangalism. He can be found on the road to just about anywhere to preach the good word about ADCs, Application Aware networking, Network Side Scripting and geekery in general to anyone that will listen.

Colin currently helps manage and maintain DevCentral ( He is also a contributor in many ways, from Articles to Videos to numerous forum posts, to iRules coding and whatever else he can get his hands on that might benefit the community and allow it to continue to grow.

Latest Stories
In his session at 21st Cloud Expo, Michael Burley, a Senior Business Development Executive in IT Services at NetApp, described how NetApp designed a three-year program of work to migrate 25PB of a major telco's enterprise data to a new STaaS platform, and then secured a long-term contract to manage and operate the platform. This significant program blended the best of NetApp’s solutions and services capabilities to enable this telco’s successful adoption of private cloud storage and launching o...
The digital transformation is real! To adapt, IT professionals need to transform their own skillset to become more multi-dimensional by gaining both depth and breadth of a wide variety of knowledge and competencies. Historically, while IT has been built on a foundation of specialty (or "I" shaped) silos, the DevOps principle of "shifting left" is opening up opportunities for developers, operational staff, security and others to grow their skills portfolio, advance their careers and become "T"-sh...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Despite being the market leader, we recognized the need to transform and reinvent our business at Dynatrace, before someone else disrupted the market. Over the course of three years, we changed everything - our technology, our culture and our brand image. In this session we'll discuss how we navigated through our own innovator's dilemma, and share takeaways from our experience that you can apply to your own organization.
Having been in the web hosting industry since 2002, dhosting has gained a great deal of experience while working on a wide range of projects. This experience has enabled the company to develop our amazing new product, which they are now excited to present! Among dHosting's greatest achievements, they can include the development of their own hosting panel, the building of their fully redundant server system, and the creation of dhHosting's unique product, Dynamic Edge.
Cloud Storage 2.0 has brought many innovations, including the availability of cloud storage services that are less expensive and much faster than previous generations of cloud storage. Cloud Storage 2.0 has also delivered new and faster methods for migrating your premises storage environment to the cloud and the concept of multi-cloud. This session will provide technical details on Cloud Storage 2.0 and the methods used to efficiently migrate from premises-to-cloud storage. This session will als...
DXWorldEXPO LLC announced today that Nutanix has been named "Platinum Sponsor" of CloudEXPO | DevOpsSUMMIT | DXWorldEXPO New York, which will take place November 12-13, 2018 in New York City. Nutanix makes infrastructure invisible, elevating IT to focus on the applications and services that power their business. The Nutanix Enterprise Cloud Platform blends web-scale engineering and consumer-grade design to natively converge server, storage, virtualization and networking into a resilient, softwar...
The Transparent Cloud-computing Consortium (T-Cloud) is a neutral organization for researching new computing models and business opportunities in IoT era. In his session, Ikuo Nakagawa, Co-Founder and Board Member at Transparent Cloud Computing Consortium, will introduce the big change toward the "connected-economy" in the digital age. He'll introduce and describe some leading-edge business cases from his original points of view, and discuss models & strategies in the connected-economy. Nowad...
For far too long technology teams have lived in siloes. Not only physical siloes, but cultural siloes pushed by competing objectives. This includes informational siloes where business users require one set of data and tech teams require different data. DevOps intends to bridge these gaps to make tech driven operations more aligned and efficient.
All in Mobile is a mobile app agency that helps enterprise companies and next generation startups build the future of digital. We offer mobile development and design for smartphones, tablets and wearables. Our projects cover the latest and most innovative technologies - voice assistants, AI, AR/VR and more. We excel at solutions for sports, fintech and retail industries.
NanoVMs is the only production ready unikernel infrastructure solution on the market today. Unikernels prevent server intrusions by isolating applications to one virtual machine with no users, no shells and no way to run other programs on them. Unikernels run faster and are lighter than even docker containers.
The dream is universal: heuristic driven, global business operations without interruption so that nobody has to wake up at 4am to solve a problem. Building upon Nutanix Acropolis software defined storage, virtualization, and networking platform, Mark will demonstrate business lifecycle automation with freedom of choice and consumption models. Hybrid cloud applications and operations are controllable by the Nutanix Prism control plane with Calm automation, which can weave together the following: ...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO Silicon Valley 2019 will cover all of these tools, with the most comprehensive program and with 222 rockstar speakers throughout our industry presenting 22 Keynotes and General Sessions, 250 Breakout Sessions along 10 Tracks, as well as our signature Power Panels. Our Expo Floor will bring together the leading global 200 companies throughout the world of Cloud Computing, DevOps, IoT, Smart Cities, FinTech, Digital Transformation, and all they entail. As ...
Darktrace is the world's leading AI company for cyber security. Created by mathematicians from the University of Cambridge, Darktrace's Enterprise Immune System is the first non-consumer application of machine learning to work at scale, across all network types, from physical, virtualized, and cloud, through to IoT and industrial control systems. Installed as a self-configuring cyber defense platform, Darktrace continuously learns what is ‘normal' for all devices and users, updating its understa...
Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes. We are offering early bird savings...