WebTest brings down Visual Studio.

I started a WebTest that fires 200,000 requests to the web server. I left the test running overnight. On getting back to work today, I found that the Visual Studio instance running the test has crashed!! To my dismay, Visual Studio did not even generate a trx file, which would have given me some idea as to how many requests it was able to shoot and what might have caused it to crash.

This experience has made me wiser though. I have now kicked off the webtest using command line.

mstest /testcontainer:testproject2.dll /test:WebTest1Coded /runconfig:localtestrun.Testrunconfig /resultsfile:testResults.trx

The test has completed and I have a 639 MB trx file lying on my disk, which always brings down Visual Studio when I try to load it. So it is now as good as not having the file to analyze. Take this lesson from me (for free) - Do not try to run a WebTest that will generate a trx file of such a size that it cannot be opened later in VS for analysis. I could not find anything on msdn that tells me of the upper limit on size of trx file. Can we have that, Microsoft?

Instead of trying to find a fault with Microsoft, let me see what could have gone wrong in the code. The WebTest code was firing 200,000 requests in a for loop with each request getting a different aspx page. I think I should remove the for loop, parameterize the web page and create a load test based on this webtest. Let me check this. Will update this space in a few hours :)

Load testing Excel Data Transfer to SQL Server 2008

Ever tried to transfer 1 million rows from Excel to SQL Server using the DTS Wizard in SQL Server 2008? Well I did and here is what I observed. My file had 3 columns in it and the total size of the file on disk was 37 MB. After I specified the Excel file as the source of data and clicked the Next button on the DTS Wizard, the memory consumption of DTS Wizard went up and up till it reached 1.5 GB. It was only then that it presented me with the select destination screen and the memory footprint went down. When I was done with all the steps and started the transfer, it again touched 1.5GB at the step named "Source Connection". I waited for it to reach the step named "Copying..." but I lost my patience and clicked on the Stop button. Inspite of that, the DTSWizard continued execution and I ended up killing it from Task Manager. I will try it again some other day when I have enough time to stare at the screen or keep it running overnight. I think it would be better to write my own program that uses SQLBulkCopy with some optimizations.

Just a thought before I hit the sack. Wouldn't it be cool, if an application like DTS Wizard, would tell you "I am going to take this much memory, this much time for completing this task. Are you willing to continue?" :). If I select not to continue, will it present me with a list of alternatives to do the same task and direct me to a site that gets me in touch with the smart minds @ MS. Any takers at Microsoft?

ASP.NET SQLCacheDependency challenging scenarios

There is an ASP.NET application using VirtualPathProvider to return web pages from the database. The VPP uses SQL Server Query Notification (SQLCacheDependency) to get notified if pages change in the database. Hence for every web page request, an entry goes into the database registering w3wp.exe, which handles the page request, as a subsriber to page change notifications.

Lets assume that the application has handled 1 million requests in a day. So there are 1 million subscriptions in the DB registered against the process id of w3wp.exe (pid:1018). Now lets take a look at the list of scenarios which must be considered by the development team.
Case 1: Application pool recycled and then Page is changed in the DB
The subscriber process w3 with pid 1018 disappears. SQL Server will attempt to send a change notification but will fail and log the error message. User is very likely to see an older version of the page. Disk space utilization will increase.

Case 2: SQL Server recycled.
SQL Server will attempt to send out 1 million change notifications to the w3wp.exe process with pid 1018. Memory utilization of SQL Server will shoot up. Users may experience slow applications response.

Case 3: SQL Server and App pool recycled.
SQL Server will error out when trying to send out change notifications to the w3wp.exe process. Memory utilization of SQL Server & disk space utilization will shoot up. Users may experience slow applications response.

All 3 cases present an interesting engineering problem to solve! Will blog more about the possible solutions. Keep watching this space.

IIS LogonMethod property

I am trying to figure out what the LogonMethod Property associated with an Application Pool means(/LM/W3SVC/AppPools, /LM/W3SVC/AppPools/DefaultAppPool/application_pool_name) means. Based on what I have found so far, I am posting some notes.

We associate credentials of a domain or local account with an application pool. Before a w3wp.exe process is started using the credentials of this account, the system calls the LogonUser function. This function accepts a parameter named "dwLogonType". I think that the value of this parameter is set based on the value assigned to the LogonMethod property in IIS Metabase. By default, IIS 6.0 uses NETWORK_CLEARTEXT as the value for the LogonMethod property. If the function succeeds in authenticating the user, it returns a handle to a token that represents the app pool account and a w3wp process is created to run in the context of this account. When using NETWORK_CLEARTEXT as the value for the LogonMethod property, the username and password of the app pool account are preserved in the DLL containing the authentication logic and used for establishing connections to other network resources during the process lifetime.

References:
1. David Wang's Post
2. MS Support article

IIS MaxBandWidth property

IIS MaxBandWidth property can be set to a value specified in bytes per second indicating an upper limit on the amount of bandwidth to be consumed by the service. By setting this value to a rightly calculated value, we can simulate scenarios of broadband or dial up access to the site.

For quick reference, I have reproduced the calculation steps given by Chris. Thanks Chris!
In order to accurately simulate a 56K dial-up connection, we must convert Kbits into Kbytes, remembering also to account for modem sync information. Since there are eight bits in a byte, you’d think we’d simply divide 56 by eight to determine the number of kilobytes to set the throttle to. However, don’t forget that part of the 56K bandwidth is dedicated to keeping the connection up and not used for data. As such, we actually divide 56 by 12 to arrive at the correct number. We then multiply by 1,024 to turn Kbytes into bytes. This results in 4,778 and change. You can keep the change.

Windows Ports and Services

I am engaged in creating a deployment topology for an ASP.NET application. One of the important things to consider is the list of services to run on the servers and what ports need to be kept open without compromising security of the system. I found this page on technet to be a very handy reference.

Adding a few more references:
1. List of IIS 6.0 Metabase properties. You can find the same list on your server in C:\Windows\Help\iismmc.chm.
2. System Definition Model (SDM) types that differ from their counterparts in the IIS metabase
3. How to Modify the Cache-Control HTTP Header When You Use IIS
4. Evolution of IIS architecture

.NET Runtime versions.

Aaron Stebner's blog has a table that lists all .NET versions released so far. This table is very useful if you are creating a Logical Data Center diagram in Visual Studio Team Architect and want to specify the 4 part version number of the Target Framework. Thanks Aaron! :)

Selective proxy setting in WebTests

In developing a WebTest, I came across a scenario where I wanted a set of requests to use a proxy and another set of requests to bypass the proxy. A WebTest has a property named Proxy as shown in the figure below, which will be used by the Visual Studio Test Engine when sending out Http Requests.
In my WebTest, I wanted the 1st two WebRequests to bypass the proxy and the 3rd one to use a Proxy. Here I found it useful to create a WebTestRequestPlugin that is later associated with a WebRequest to set a proxy in the PreRequest method.


Here is the code for the WebTestRequestPlugin.

public class ProxySetter : WebTestRequestPlugin

{

    string proxy = null;

 

    public string Proxy

    {

        get

        {

            return this.proxy;

        }

        set

        {

            this.proxy = value;

        }

    }

 

    public override void PreRequest(object sender, PreRequestEventArgs e)

    {

        if (!String.IsNullOrEmpty(this.proxy))

        {

            e.WebTest.Proxy = this.proxy;

        }

        else

        {

            e.WebTest.Proxy = null;

        }

    }

}