SYS-CON MEDIA Authors: Yeshim Deniz, Elizabeth White, Pat Romanski, Liz McMillan, Courtney Abud

Blog Feed Post

HTTP Request Cloning via iRules, Part 1

One of the requests that I've seen several times over the years is the ability to completely clone web requests across multiple servers. The idea is that you can take the HTTP traffic coming in bound for pool member A and distribute it, in its entirety, to pool member B. Or perhaps members B-G..whatever your needs are. This can be helpful for many reasons, security auditing, test or dev harnesses, archival, etc. Whatever the reasons, this has been a repeated question in the forums and in the field. While clone pool functionality works to some degree for this, it doesn't work quite as desired, and doesn't easily distribute to multiple additional members.

iRules, however, offers a solution.

Using the HSL feature in iRules that, if you remember, allows you to specify a protocol and destination, which can be a pool, we are able to treat this much like sideband connections in v11. By establishing a new connection and sending across the HTTP info as needed we're able to clone the HTTP traffic in its entirety. Let's take a look at how this starts:

   1: when CLIENT_ACCEPTED {
   2:     # Open a new HSL connection if one is not available
   3:     set hsl [HSL::open -proto TCP -pool http_clone_pool]
   4:     log local0. "[IP::client_addr]:[TCP::client_port]: New hsl: $hsl"
   5: }

As you can see, it's straight-forward enough. Using the HSL::open command we set the protocol to TCP and the pool to whichever pool you'd like to clone your HTTP traffic to. Now that we know where and how we're sending the data, we need to figure out which data to send. The only trick with HTTP in this step is that GET and POST requests need to be handled differently. With a POST we will need to collect the data that is being posted so that we can replay it back to the new destination. With a GET we simply forward through the headers of the request. Fortunately determining which is which is a cake walk in iRules, so it's just the collecting and forwarding bit we really need to worry about. This is the real "meat" of this iRule, and even that isn't difficult, it looks like:

   1: when HTTP_REQUEST {
   2:     # Insert an XFF header if one is not inserted already
   3:     # So the client IP can be tracked for the duplicated traffic
   4:     HTTP::header insert X-Forwarded-For [IP::client_addr]
   6:     # Check for POST requests
   7:     if {[HTTP::method] eq "POST"}{
   9:         # Check for Content-Length between 1b and 1Mb
  10:         if { [HTTP::header Content-Length] >= 1 && [HTTP::header Content-Length] < 1048576 }{
  11:             HTTP::collect [HTTP::header Content-Length]
  12:         } elseif {[HTTP::header Content-Length] == 0}{
  13:             # POST with 0 content-length, so just send the headers
  14:             HSL::send $hsl [HTTP::request]
  15:             log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  16:         }
  17:     } else {
  18:         # Request with no payload, so send just the HTTP headers to the clone pool
  19:         HSL::send $hsl [HTTP::request]
  20:         log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  21:     }
  22: }

As you can see this is pretty standard iRules fare for the most part. HTTP::method, HTTP::header, HTTP::collect. Nothing shocking for the most part. The real trick is in the HSL::send command. Note that it's going to "$hs1"? That's the connection we established earlier with the HSL::open command. Now that we have that handle available we're able to easily forward through other traffic. So as you can see in the POSTs with content attached  we're collecting, and anything else we're forwarding along the headers alone. Note that nothing has been sent for the POSTs that have content attached, we've just entered a collect state so the client will continue sending data and we'll store it. That data is then available in the HTTP_REQUEST_DATA event, and we can forward it along when that occurs.  So for those particular requests an additional event will fire:

   1: when HTTP_REQUEST_DATA {
   2:     # The parser does not allow HTTP::request in this event, but it works
   3:     set request_cmd "HTTP::request"
   4:     log local0. "[IP::client_addr]:[TCP::client_port]: Collected [HTTP::payload length] bytes,\
   5:         sending [expr {]string length [eval $request_cmd][ + ]HTTP::payload length[}] bytes total"
   6:     HSL::send $hsl "[eval $request_cmd][HTTP::payload]"
   7: }

Now that the HTTP_REQUEST_DATA event has fired we know our collect has picked up the data we want it to. This event will only fire after a successful HTTP::collect. Once this happens we're ready to forward along the POST and the accompanying data.  After a little expr trickery to convince the parser to allow the HTTP::request command within the HTTP_REQUEST_DATA event (it doesn't think it should work, but it we trick it) we're able to send along the original request and payload data without much hassle. Again making use of the HSL::send command and the $hs1 variable we set up at the beginning makes this process easy.

At this point you now have a functioning iRule that will clone traffic inbound for your Virtual to another pool of your choosing. At this point you are probably asking yourself three questions.

1) Why hasn't this been written before?

2) Where is the version that allows forwarding to multiple other pools?!

3) Why HSL and not sideband connections?

Well, those answers are simple:

1) Because our good friend Hoolio hadn't written it yet! Aaron whipped this together and posted it. I got his okay to write it up and get it out there, so here it is. Keep in mind that this is VERY early in the testing stages and is prone to change/update. I'm sharing it here because I think it's awesome, and don't want it to slip off into the night without being called out. But this is very much a use at your own risk sort of thing for now. I'll update with notes when more testing has been done. Also worth note is that this requires at least version 10.1 or newer to function.

2) It's coming, don't fret. That will be Part 2! You did notice the Part1 in the title didn't you? We can't give it all away at once. Besides that part is still under testing. Releasing it before it's ready wouldn't be prudent. Stay tuned, it's coming.

3) I asked Aaron the exact same thing and here's what he said:

HSL automatically ACKs the server responses, but ignores the data.  From limited testing of both HSL and sideband connections, HSL is also a lot more efficient in handling high connection rates.  Also, HSL is available from 10.1 and sideband only on 11.x.

So there you have it. Sideband connections would work just fine, but HSL allows for a wider audience (10.1 and above), and offers a little added efficiency/ease of use in this particular case. Keep in mind that HSL won't handle many of the more complex scenarios that sideband connections will, hence the tradeoff, but in this particular case HSL seems to win out.

Keep an eye out for the next installment of this two-parter, wherein you'll see how to extend this model to work with multiple clone destinations. For now, any questions/comments are welcome. If you get this up and running in test/dev, please let us know as we'd love any feedback resulting from that testing. Many thanks to hoolio for his continued, outstanding contributions. With that, I'll leave you with the full version of the iRule:

   1: when CLIENT_ACCEPTED {
   2:     # Open a new HSL connection if one is not available
   3:     set hsl [HSL::open -proto TCP -pool http_clone_pool]
   4:     log local0. "[IP::client_addr]:[TCP::client_port]: New hsl: $hsl"
   5: }
   6: when HTTP_REQUEST {
   8:     # Insert an XFF header if one is not inserted already
   9:     # So the client IP can be tracked for the duplicated traffic
  10:     HTTP::header insert X-Forwarded-For [IP::client_addr]
  12:     # Check for POST requests
  13:     if {[HTTP::method] eq "POST"}{
  15:         # Check for Content-Length between 1b and 1Mb
  16:         if { [HTTP::header Content-Length] >= 1 && [HTTP::header Content-Length] < 1048576 }{
  17:             HTTP::collect [HTTP::header Content-Length]
  18:         } elseif {[HTTP::header Content-Length] == 0}{
  19:             # POST with 0 content-length, so just send the headers
  20:             HSL::send $hsl [HTTP::request]
  21:             log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  22:         }
  23:     } else {
  24:         # Request with no payload, so send just the HTTP headers to the clone pool
  25:         HSL::send $hsl [HTTP::request]
  26:         log local0. "[IP::client_addr]:[TCP::client_port]: Sending [HTTP::request]"
  27:     }
  28: }
  29: when HTTP_REQUEST_DATA {
  30:     # The parser does not allow HTTP::request in this event, but it works
  31:     set request_cmd "HTTP::request"
  32:     log local0. "[IP::client_addr]:[TCP::client_port]: Collected [HTTP::payload length] bytes,\
  33:         sending [expr {]string length [eval $request_cmd][ + ]HTTP::payload length[}] bytes total"
  34:     HSL::send $hsl "[eval $request_cmd][HTTP::payload]"
  35: }

Read the original blog entry...

More Stories By Colin Walker

Coming from a *Nix Software Engineering background, Colin is no stranger to long hours of coding, testing and deployment. His personal experiences such as on-stage performance and the like have helped to foster the evangelist in him. These days he splits his time between coding, technical writing and evangalism. He can be found on the road to just about anywhere to preach the good word about ADCs, Application Aware networking, Network Side Scripting and geekery in general to anyone that will listen.

Colin currently helps manage and maintain DevCentral ( He is also a contributor in many ways, from Articles to Videos to numerous forum posts, to iRules coding and whatever else he can get his hands on that might benefit the community and allow it to continue to grow.

Latest Stories
Isomorphic Software is the global leader in high-end, web-based business applications. We develop, market, and support the SmartClient & Smart GWT HTML5/Ajax platform, combining the productivity and performance of traditional desktop software with the simplicity and reach of the open web. With staff in 10 timezones, Isomorphic provides a global network of services related to our technology, with offerings ranging from turnkey application development to SLA-backed enterprise support. Leadin...
Take advantage of autoscaling, and high availability for Kubernetes with no worry about infrastructure. Be the Rockstar and avoid all the hurdles of deploying Kubernetes. So Why not take Heat and automate the setup of your Kubernetes cluster? Why not give project owners a Heat Stack to deploy Kubernetes whenever they want to? Hoping to share how anyone can use Heat to deploy Kubernetes on OpenStack and customize to their liking. This is a tried and true method that I've used on my OpenSta...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Kubernetes is a new and revolutionary open-sourced system for managing containers across multiple hosts in a cluster. Ansible is a simple IT automation tool for just about any requirement for reproducible environments. In his session at @DevOpsSummit at 18th Cloud Expo, Patrick Galbraith, a principal engineer at HPE, will discuss how to build a fully functional Kubernetes cluster on a number of virtual machines or bare-metal hosts. Also included will be a brief demonstration of running a Galer...
10ZiG Technology is a leading provider of endpoints for a Virtual Desktop Infrastructure environment. Our fast and reliable hardware is VMware, Citrix and Microsoft ready and designed to handle all ranges of usage - from task-based to sophisticated CAD/CAM users. 10ZiG prides itself in being one of the only companies whose sole focus is in Thin Clients and Zero Clients for VDI. This focus allows us to provide a truly unique level of personal service and customization that is a rare find in th...
Emil Sayegh is an early pioneer of cloud computing and is recognized as one of the industry's true veterans. A cloud visionary, he is credited with launching and leading the cloud computing and hosting businesses for HP, Rackspace, and Codero. Emil built the Rackspace cloud business while serving as the company's GM of the Cloud Computing Division. Earlier at Rackspace he served as VP of the Product Group and launched the company's private cloud and hosted exchange services. He later moved o...
92% of enterprises are using the public cloud today. As a result, simply being in the cloud is no longer enough to remain competitive. The benefit of reduced costs has normalized while the market forces are demanding more innovation at faster release cycles. Enter Cloud Native! Cloud Native enables a microservices driven architecture. The shift from monolithic to microservices yields a lot of benefits - but if not done right - can quickly outweigh the benefits. The effort required in monitoring,...
As you know, enterprise IT conversation over the past year have often centered upon the open-source Kubernetes container orchestration system. In fact, Kubernetes has emerged as the key technology -- and even primary platform -- of cloud migrations for a wide variety of organizations. Kubernetes is critical to forward-looking enterprises that continue to push their IT infrastructures toward maximum functionality, scalability, and flexibility. As they do so, IT professionals are also embr...
Signs of a shift in the usage of public clouds are everywhere. Previously, as organizations outgrew old IT methods, the natural answer was to try the public cloud approach; however, the public platform alone is not a complete solution. Complaints include unpredictable/escalating costs and mounting security concerns in the public cloud. Ultimately, public cloud adoption can ultimately mean a shift of IT pains instead of a resolution. That's why the move to hybrid, custom, and multi-cloud will ...
The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get tailored market studies; and more.
Docker is sweeping across startups and enterprises alike, changing the way we build and ship applications. It's the most prominent and widely known software container platform, and it's particularly useful for eliminating common challenges when collaborating on code (like the "it works on my machine" phenomenon that most devs know all too well). With Docker, you can run and manage apps side-by-side - in isolated containers - resulting in better compute density. It's something that many developer...
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
The KCSP program is a pre-qualified tier of vetted service providers that offer Kubernetes support, consulting, professional services and training for organizations embarking on their Kubernetes journey. The KCSP program ensures that enterprises get the support they're looking for to roll out new applications more quickly and more efficiently than before, while feeling secure that there's a trusted and vetted partner that's available to support their production and operational needs.
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It's clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Th...
xMatters helps enterprises prevent, manage and resolve IT incidents. xMatters industry-leading Service Availability platform prevents IT issues from becoming big business problems. Large enterprises, small workgroups, and innovative DevOps teams rely on its proactive issue resolution service to maintain operational visibility and control in today's highly-fragmented IT environment. xMatters provides toolchain integrations to hundreds of IT management, security and DevOps tools. xMatters is the ...