HttpWebRequestWrapper is a testing layer for Microsoft's HttpClient, HttpWebRequest and WebClient classes. It overcomes restrictions that would normally prevent mocking a HttpWebRequest and allows testing your application with faked HTTP requests in Unit and BDD tests.
HttpWebRequestWrapper is built with some serious secret sauce allowing you to intercept nearlly all http traffic, including calls made in 3rd party code and code that doesn't support dependency injection! It's the ideal testing tool for testing application code that relies on http api calls either directly or through 3rd party libraries!
PM> Install-Package HttpWebRequestWrapper
HttpWebRequestWrapper has no 3rd Party dependencies!
// Asserts we can use a HttpWebRequestWrapperSession to // intercept and replace the Response to // new HttpClient().GetStringAsync("https://www.github.com")[Test]publicasyncTaskInterceptAndReplaceTrafficToGitHub(){varfakeResponse="Testing";varinterceptor=newHttpWebRequestWrapperInterceptorCreator(x =>x.HttpWebResponseCreator.Create(fakeResponse));using(newHttpWebRequestWrapperSession(interceptor)){varresponseBody=awaitnewHttpClient().GetStringAsync("https://www.github.com");Assert.Equal(fakeResponse,responseBody);}}Let's say you have some simple but very hard to test code that uses WebRequest.Create to make a live http call:
publicstaticclassExample{publicstaticintCountCharactersOnAWebPage(stringurl){// makes live network callvarresponse=(HttpWebResponse)WebRequest.Create(url).GetResponse();if(response.StatusCode!=HttpStatusCode.Ok)thrownewException();using(varsr=newStreamReader(response.GetResponseStream())
return sr.ReadToEnd().Length;}}It would be ideal if this code used seperation of concerns and dependency injection to be more testable. But perhaps it's in a 3rd party library, or it'll be too expensive to refacotr.
Fortunatly, it can b easily unit tested with the HttpWebRequestWrapper Library! The test below intercepts the http call made by Example.CountCharactersOnAWebPage and returns a fake html response:
// ARRANGEvarfakeResponseBody="<html>Test</html>";using(newHttpWebRequestWrapperSession(newHttpWebRequestWrapperInterceptorCreator(req =>req.HttpWebResponseCreator.Create(fakeResponseBody)))){// ACT varcharactersOnGitHub=Example.CountCharactersOnAWebPage("http://www.github.com");// ASSERTAssert.Equal(fakeResponseBody.Length,charactersOnGitHub);}HttpWebRequestWrapper fully supports the .net HttpClient and WebClient classes as well as the static WebRequest.Create() method:
varfakeResponse="Testing";using(newHttpWebRequestWrapperSession(newHttpWebRequestWrapperInterceptorCreator(
x =>x.HttpWebResponseCreator.Create(fakeResponse)))){varresponseBody1=awaitnewHttpClient().GetStringAsync("https://www.github.com");varresponseBody2=newWebClient().DownloadString("https://www.github.com");varresponse=WebRequest.Create("https://www.github.com").GetResponse();Assert.Equal(fakeResponse,responseBody1);Assert.Equal(fakeResponse,responseBody2);using(varsr=newStreamReader(response.GetResponseStream())
Assert.Equal(fakeResponse,sr.ReadToEnd());}Use the HttpWebRequestWrapperRecorder to capture all http requests and response into a serializable RecordingSession for later playback in reliable and consistent tests!
// run the application using the recordervarrecordingSession=newRecordingSession();using(newHttpWebRequestWrapperSession(newHttpWebRequestWrapperRecorderCreator(recordingSession))){Example.CountCharactersOnAWebPage("http://www.github.com");}// serialize the recordingSession to disk (and preferrably embed in test assembly)File.WriteAllText(@"c:\recordingSession.json",JsonConvert.Serialize(recordingSession));Next, deserialize the recording session json and use the RecordingSessionInterceptorRequestBuilder to feed the RecordingSession into the HttpWebRequestWrapperInterceptor
varrecordingSession=JsonConvert.DeserializeObject<RecordingSession>(json);using(newHttpWebRequestWrapperSession(newHttpWebRequestWrapperInterceptorCreator(newRecordingSessionInterceptorRequestBuilder(recordingSession)))){// now no http calls!varcount=Example.CountCharactersOnAWebPage("http://www.github.com");}Assert.AreEqual(4321,count);RecordingSessionInterceptorRequestBuilder exposes multiple customization points to override how matching is performed, how responses are built, and what to do if a match can't be found:
varrecordingSession=JsonConvert.DeserializeObject<RecordingSession>(json);newRecordingSessionInterceptorRequestBuilder(recordingSession){MatchingAlgorithm=(interceptedRequest,recordedRequet)=>// match only on urlinterceptedReq.HttpWebRequest.RequestUri==recordedRequet.Url,RecordedResultResponseBuilder=(recordedReq,interceptedReq)=>// manipulate the response bodyinterceptedReq.HttpWebResponseCreator.Create(recordedRequest.Response.ToLower()),RequestNotFoundResponseBuilder= interceptedReq =>// return a 500 when page isn't foundinterceptedReq.HttpWebResponseCreator.Create("Server Error",HttpStatusCode.InternalServerError),OnMatch=(recordedReq,interceptedReq,httpWebResponse,exception)=>// keep a count of requests made or do additional manipulation of the web responseLog.Write("Application made another request");AllowReplayingRecordedRequestsMultipleTimes=// act like a playback script - each recorded request in the recording // session will only be used one - useful for testing error handling / retry// behaviorfalse};But if that's not enough, you can easily implement your own IInterceptorRequestBuilder for full control!
Any Exception thown during HttpWebRequest.GetResponse() is captured by HttpWebRequestWrapperRecorder and is rethrown by RecordingSessionInterceptorRequestBuilder during playback! This includes a fully populated WebException with a WebException.Response.
Additionally, RequestNotFoundResponseBuilder also supports throwing exceptions, you're free to overload and throw a NotFound WebException.
Want to change up playback mid test run? Not a problem! You can easily manipulate RecordingSessionInterceptorRequestBuilder.RecordedRequests at any time:
varbuilder=newRecordingSessionInterceptorRequestBuilder();builder.RecordedRequests.Add(newRecordedRequest{Url="http://www.github.com",Method="GET",Response="Hello World"};using(newHttpWebRequestWrapperSession(newHttpWebRequestWrapperInterceptorCreator(builder))){varcount1=Example.CountCharactersOnAWebPage("http://www.github.com");builder.RecordedRequests.Clear();builder.RecordedRequests.Add(newRecordedRequest{Url="http://www.github.com",Method="GET",Response="!!!"});varcount2=Example.CountCharactersOnAWebPage("http://www.github.com");}Assert.AreEqual(count1,count2);Have a lot of network requests recorded? Split them up over multiple recording sessions!
varrecordingSession1=JsonConvert.DeserializeObject<RecordingSession>(json1);varrecordingSession2=JsonConvert.DeserializeObject<RecordingSession>(json2);using(newHttpWebRequestWrapperSession(newHttpWebRequestWrapperInterceptorCreator(newRecordingSessionInterceptorRequestBuilder(recordingSession1,recordingSession2))))Inspired by HttpWebRequestWrapperInterceptor and HttpWebRequestWrapperRecorder and want to build your own custom http request wrapper? Not a problem, add an IWebRequestCreate to go with it and you'll be good to use HttpWebRequestWrapperSession to take care of the plumbing for you:
publicclassCustomWrapper:HttpWebRequestWrapper{publicCustomWrapper(Uriuri):base Uri{}}publicclassCustomWrapperCreator:IWebRequestCreate{publicWebRequestCreate(Uriuri){returnnewHttpWebRequestWrapperInterceptor(uri);}}using(newHttpWebRequestWrapperSession(newCustomWrapperCreator())){varrequest=WebRequest.Create("http://www.github.com");Assert.IsType<CustomWrapper>(request);}Have an advanced scenario where you need to use multiple IWebRequestCreate objects? You can use the HttpWebRequestWrapperDelegateCreator to decide just-in-time which IWebRequestCreate to use for a specific Uri:
varcreatorSelector=newFunc<Uri,IWebRequestCreate>(url =>url.Contains("api1")?api1InterceptorCreator:commonInterceptorCreator);using(newHttpWebRequestWrapperSession(newHttpWebRequestWrapperDelegateCreator(creatorSelector))){// handled by api1InterceptorWebRequest.Create("http://3rdParty.com/api1/request");// handled by commonInterceptorWebRequest.Create("http://someother.site");}Go crazy with full mocking support! The HttpWebRequestWrapperInterceptor provides very powerful faking, but you can easily build your own mock HttpWebRequestWrapper to provide custom behavior or expectations.
// ARRANGEvarfakeResponseBody="Fake Response";varmockWebRequest=newMock<HttpWebRequestWrapper>(newUri("http://www.github.com"));mockWebRequest.Setup(x =>x.GetResponse()).Returns(HttpWebResponseCreator.Create(newUri("http://www.github.com"),"GET",HttpStatusCode.OK,"Fake Response"));varmockCreator=newMock<IWebRequestCreate>();mockCreator.Setup(x =>x.Create(It.IsAny<Uri>())).Returns(mockWebRequest.Object);// ACTstringresponseBody;using(newHttpWebRequestWrapperSession(mockCreator.Object)){request=(HttpWebRequest)WebRequest.Create("http://www.github.com");using(varsr=newStreamReader(request.GetResponse().GetResponseStream()))responseBody=sr.ReadToEnd();}// ASSERTAssert.Equal(fakeResponseBody,responseBody);mockWebRequest.Verify(x =>x.GetResponse());HttpWebRequestWrapper works by inheriting from HttpWebRequest. This doesn't seem revolutionary, except these are the HttpWebRequest constructors:
[Obsolete("This API supports the .NET Framework infrastructure and is not intended to be used directly from your code.",true)]publicHttpWebRequest(){}internalHttpWebRequest(Uri uri,ServicePoint servicePoint){// bunch of initialization code}Niether constructor will allow compilation of code that tries to inherit from HttpWebRequest.
Instead HttpWebRequestWrapper is compiled to inherit from WebRequest. During build, the compiled assembly is deassembled, has it's IL manipulated so that it inherits from HttpWebRequest and is then reasembled. HttpWebRequestWrapper also does a bunch of reflection inside its constructor so the end result is a public class with a public constructor that inherits from HttpWebRequest and is fully functional!
The second piece of magic is hooking into WebRequest.PrefixList. WebRequest works as a factory for factories. The PrefixList contains the registration of which factory is matched to which protocol. HttpWebRequestWrapperSession works by overriding the default PrefixList, replacing the IWebRequestCreate for http and https.
Disposing of the Session restores the original Prefix list.
HttpClient uses an internal HttpWebRequest constructor to directly create requests and by-passes WebRequest.PrefixList. So instead,
HttpWebRequestWrapper uses a custom TaskScheduler to hook into HttpClientHandler and hijack the HttpClientHandler.StartRequeset method to replace the underlying HttpWebRequest it will use with one created via WebRequest.Create.
HttpWebRequestWrapper can not support concurrent test execution. Becuase HttpWebRequestWrapperSession works by setting a global static variable it's not possible to have two Sessions in use at one time. Code inside the Session's using block is free to execute concurrently, you just can't try and use two or more Sessions at once.
HttpClient performs IO via the abstract HttpMessageHandler. By default, this is a HttpClinetHandler. HttpWebRequestWrapper fully supports HttpClientHandler and any handler that inherits from HttpClientHandler. Custom handlers that instead inheirt directly from HttpMessageHandler are not supported and will not be intercepted.
The actual wrapper HttpWebRequestWrapper is compiled for .NET Framework 2.0 and supports up to versions 4.7 of the .NET Framework (newest version tested at the time).
The remainder of HttpWebRequestWrapper requires .NET Framework 3.5+
Support for HttpClient requires .NET Framework 4.5+
Clone the repository and build /src/HttpWebRequestWrapper.sln using Visual Studio. NuGet package restore must be enabled.
To generate a nuget package run:
nuget pack .\src\HttpWebRequestWrapper\HttpWebRequestWrapper.csproj
NOTE: I recommend unloading the HttpWebRequestWrapper.LowLevel project after opening the solution in Visual Studio and building for the first time. The IDE (and tools like ReSharper) get confused by HttpWebRequestWrapper inheriting from WebRequest in source code and HttpWebRequest in the built dll. Consequence is a lot of warnings and false errors.