InfoCard download issue & response caching.

I have been studying the Zermatt code samples over the weekend in order to see how I can issue a managed card to an authenticated user of my web application that uses Windows Live Id authentication service. So after a user is authenticated, there is a button to be clicked to download the infocard. When I tested this on Firefox 3.0, the download worked fine. But on IE 7, I got an error message.

---------------------------
Windows Internet Explorer
---------------------------
Internet Explorer cannot download signin.aspx from localhost. Internet Explorer was not able to open this Internet site. The requested site is either unavailable or cannot be found. Please try again later.
---------------------------
OK
---------------------------

This was happening because of the following statement in Page Load event
Response.Cache.SetCacheability(HttpCacheability.NoCache);
was causing the download of the infocard to fail.

What does this statement do?
With this statement in the code, the response sent back to the client has "Cache-Control: no-cache" in the header.

Why should this error occur on IE, but not firefox?
IE, probably, handles "no-cache" setting without errors when the response is "text/html" encoded. In my case, I was returning a "application/x-informationcardfile" in the response and I guess IE threw up the error message since it could not find how not to cache this response.
Firefox 3.0 just ignores the cache-control HTTP header.

I replaced the offending statement with
Response.Cache.SetNoStore();
and the file download works fine on both IE & Firefox.

References:
1. File download problem with Response.Cache.SetCacheability
2. Cache directives ignored by Firefox in ASP.NET caching.

CardSpace STS error: Certificate used has a trust chain that cannot be verified

When trying to run the Simple STS sample application, I came across this message.

The certificate used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. The revocation function was unable to check revocation because the revocation server was offline.


On further investigation, I realized that the revocation server was set to http://www.adatum.com in the sample certificate. I had an entry for http://www.adatum.com in the Hosts file on my dev box. But it was not configured under IE Options > Connections > LAN Settings > Advanced > Proxy Settings > Exceptions. On adding the missing entry in the Exceptions text box, the error was fixed! Here is a list of commor errors when running cardspace samples.

SSL setup issue in Zermatt Beta.

I downloaded Microsoft Zermatt Beta and was enthusiastically working to get the samples setup. When running the script SamplesPreReqSetup.bat, I got the following error in the SSL setup step.

C:\Program Files\Microsoft Code Name Zermatt\Samples\Utilities\Scripts\SetSSLCertificate.js(32, 1) (null): The system cannot find the path specified.

On having a look at the script, I found that the code was not able to find the path "/W3SVC/1" in the metabase.xml. The setup script assumes that the default web site is available at the path /W3SVC/1. On opening up metabase.xml, I found that the Default Web Site on my server was set to "/LM/W3SVC/998577302". I guess that the Default Web Site was probably deleted and reconfigured on this box. Hence, it had a different value for the Location attribute. :). As a fix for this, I have replaced the following line in SamplesPreReqCleanup.bat
cscript //nologo "%~dp0Scripts\SetSSLCertificate.js" /W3SVC/1 %localhostCERTHASH%

with the one below.
cscript //nologo "%~dp0Scripts\SetSSLCertificate.js" /W3SVC/998577302 %localhostCERTHASH%

How to fix "Theme cannot be found errors"

Wiith the following Website structure.
- Default Web Site
  - vdir1
    - App_Code
  - App_Code
  - App_Themes
    - GlobalTheme

when I queried for a page http://localhost/vdir1/Test.aspx, I got the following error message "ConfigurationErrorsException: Theme 'GlobalTheme' cannot be found in the application or global theme directories". The fix for this was simple. I had to create a sub-folder named App_Themes under vdir1 as shown below.

- Default Web Site
  - vdir1
    - App_Code
    - App_Themes
      - GlobalTheme
  - App_Code
  - App_Themes
    - GlobalTheme

Where is HttpCfg?

I downloaded a few samples for trying out Windows CardSpace. One of the setup scripts used HttpCfg to configure SSL and it failed to execute since it could not find HttpCfg. This file, if not on your system, can be installed from [Windows 2003 Setup location]\Support\Tools\SUPTOOLS.MSI. For a complete list of support tools click here.

Windows XP users can download the support tools from here
Windows 2003 users can download support tools from here

Random page selection for Load Testing

How should I design a load test for a scenario where a user can randomly hit one page from a given set of unsecure pages maintained on my web server? So at any point of time there could be 'x' no. of users, each accessing one page from the 'y' no. of pages residing on my web server. First I wrote a web test that will simulate 1 user accessing a random page. To do this, I created a table named a single column table named "Files" and populated it with the names of the aspx pages I have on the web server. Then I created a view over it with a SQL query

SELECT TOP(1) 'http://localhost/' + [dbo].[Files].[FileName] AS [PageName] FROM [dbo].[Files] ORDER BY NEWID()

Thanks to Philipe for giving this tip!

The above query will return a random page Url like http://localhost/Page1.aspx which is bound to a request in the WebTest.
Then I associated this WebTest with the LoadTest and configured the LoadTest to start with 1 user and add a new user every 1 second till a maximum count which will be equal to the number of pages on my web server. By selecting a distribution model based on the number of virtual users, I expected that the load test will result in user's getting a random unique page in each iteration. However, I was baffled to see what the load test engine was actually doing. On running the load test, the Page Results section in the Summary showed me that only one page was requested during the entire execution!!! On further investigation, I noticed that the WebLoadTestRequestMap table in the Load Test Database had only 1 RequestUri corresponding to my LoadTestRunId.

To get over this issue, I found it best to execute the above SQL statement using SqlDataReader in the WebTest code and create a WebTestRequest instance with the Url returned. With this approach, I am now able to successfully load test the scenario I had in mind. :)

WebTest brings down Visual Studio.

I started a WebTest that fires 200,000 requests to the web server. I left the test running overnight. On getting back to work today, I found that the Visual Studio instance running the test has crashed!! To my dismay, Visual Studio did not even generate a trx file, which would have given me some idea as to how many requests it was able to shoot and what might have caused it to crash.

This experience has made me wiser though. I have now kicked off the webtest using command line.

mstest /testcontainer:testproject2.dll /test:WebTest1Coded /runconfig:localtestrun.Testrunconfig /resultsfile:testResults.trx

The test has completed and I have a 639 MB trx file lying on my disk, which always brings down Visual Studio when I try to load it. So it is now as good as not having the file to analyze. Take this lesson from me (for free) - Do not try to run a WebTest that will generate a trx file of such a size that it cannot be opened later in VS for analysis. I could not find anything on msdn that tells me of the upper limit on size of trx file. Can we have that, Microsoft?

Instead of trying to find a fault with Microsoft, let me see what could have gone wrong in the code. The WebTest code was firing 200,000 requests in a for loop with each request getting a different aspx page. I think I should remove the for loop, parameterize the web page and create a load test based on this webtest. Let me check this. Will update this space in a few hours :)

Load testing Excel Data Transfer to SQL Server 2008

Ever tried to transfer 1 million rows from Excel to SQL Server using the DTS Wizard in SQL Server 2008? Well I did and here is what I observed. My file had 3 columns in it and the total size of the file on disk was 37 MB. After I specified the Excel file as the source of data and clicked the Next button on the DTS Wizard, the memory consumption of DTS Wizard went up and up till it reached 1.5 GB. It was only then that it presented me with the select destination screen and the memory footprint went down. When I was done with all the steps and started the transfer, it again touched 1.5GB at the step named "Source Connection". I waited for it to reach the step named "Copying..." but I lost my patience and clicked on the Stop button. Inspite of that, the DTSWizard continued execution and I ended up killing it from Task Manager. I will try it again some other day when I have enough time to stare at the screen or keep it running overnight. I think it would be better to write my own program that uses SQLBulkCopy with some optimizations.

Just a thought before I hit the sack. Wouldn't it be cool, if an application like DTS Wizard, would tell you "I am going to take this much memory, this much time for completing this task. Are you willing to continue?" :). If I select not to continue, will it present me with a list of alternatives to do the same task and direct me to a site that gets me in touch with the smart minds @ MS. Any takers at Microsoft?

ASP.NET SQLCacheDependency challenging scenarios

There is an ASP.NET application using VirtualPathProvider to return web pages from the database. The VPP uses SQL Server Query Notification (SQLCacheDependency) to get notified if pages change in the database. Hence for every web page request, an entry goes into the database registering w3wp.exe, which handles the page request, as a subsriber to page change notifications.

Lets assume that the application has handled 1 million requests in a day. So there are 1 million subscriptions in the DB registered against the process id of w3wp.exe (pid:1018). Now lets take a look at the list of scenarios which must be considered by the development team.
Case 1: Application pool recycled and then Page is changed in the DB
The subscriber process w3 with pid 1018 disappears. SQL Server will attempt to send a change notification but will fail and log the error message. User is very likely to see an older version of the page. Disk space utilization will increase.

Case 2: SQL Server recycled.
SQL Server will attempt to send out 1 million change notifications to the w3wp.exe process with pid 1018. Memory utilization of SQL Server will shoot up. Users may experience slow applications response.

Case 3: SQL Server and App pool recycled.
SQL Server will error out when trying to send out change notifications to the w3wp.exe process. Memory utilization of SQL Server & disk space utilization will shoot up. Users may experience slow applications response.

All 3 cases present an interesting engineering problem to solve! Will blog more about the possible solutions. Keep watching this space.

IIS LogonMethod property

I am trying to figure out what the LogonMethod Property associated with an Application Pool means(/LM/W3SVC/AppPools, /LM/W3SVC/AppPools/DefaultAppPool/application_pool_name) means. Based on what I have found so far, I am posting some notes.

We associate credentials of a domain or local account with an application pool. Before a w3wp.exe process is started using the credentials of this account, the system calls the LogonUser function. This function accepts a parameter named "dwLogonType". I think that the value of this parameter is set based on the value assigned to the LogonMethod property in IIS Metabase. By default, IIS 6.0 uses NETWORK_CLEARTEXT as the value for the LogonMethod property. If the function succeeds in authenticating the user, it returns a handle to a token that represents the app pool account and a w3wp process is created to run in the context of this account. When using NETWORK_CLEARTEXT as the value for the LogonMethod property, the username and password of the app pool account are preserved in the DLL containing the authentication logic and used for establishing connections to other network resources during the process lifetime.

References:
1. David Wang's Post
2. MS Support article

IIS MaxBandWidth property

IIS MaxBandWidth property can be set to a value specified in bytes per second indicating an upper limit on the amount of bandwidth to be consumed by the service. By setting this value to a rightly calculated value, we can simulate scenarios of broadband or dial up access to the site.

For quick reference, I have reproduced the calculation steps given by Chris. Thanks Chris!
In order to accurately simulate a 56K dial-up connection, we must convert Kbits into Kbytes, remembering also to account for modem sync information. Since there are eight bits in a byte, you’d think we’d simply divide 56 by eight to determine the number of kilobytes to set the throttle to. However, don’t forget that part of the 56K bandwidth is dedicated to keeping the connection up and not used for data. As such, we actually divide 56 by 12 to arrive at the correct number. We then multiply by 1,024 to turn Kbytes into bytes. This results in 4,778 and change. You can keep the change.

Windows Ports and Services

I am engaged in creating a deployment topology for an ASP.NET application. One of the important things to consider is the list of services to run on the servers and what ports need to be kept open without compromising security of the system. I found this page on technet to be a very handy reference.

Adding a few more references:
1. List of IIS 6.0 Metabase properties. You can find the same list on your server in C:\Windows\Help\iismmc.chm.
2. System Definition Model (SDM) types that differ from their counterparts in the IIS metabase
3. How to Modify the Cache-Control HTTP Header When You Use IIS
4. Evolution of IIS architecture

.NET Runtime versions.

Aaron Stebner's blog has a table that lists all .NET versions released so far. This table is very useful if you are creating a Logical Data Center diagram in Visual Studio Team Architect and want to specify the 4 part version number of the Target Framework. Thanks Aaron! :)

Selective proxy setting in WebTests

In developing a WebTest, I came across a scenario where I wanted a set of requests to use a proxy and another set of requests to bypass the proxy. A WebTest has a property named Proxy as shown in the figure below, which will be used by the Visual Studio Test Engine when sending out Http Requests.
In my WebTest, I wanted the 1st two WebRequests to bypass the proxy and the 3rd one to use a Proxy. Here I found it useful to create a WebTestRequestPlugin that is later associated with a WebRequest to set a proxy in the PreRequest method.


Here is the code for the WebTestRequestPlugin.

public class ProxySetter : WebTestRequestPlugin

{

    string proxy = null;

 

    public string Proxy

    {

        get

        {

            return this.proxy;

        }

        set

        {

            this.proxy = value;

        }

    }

 

    public override void PreRequest(object sender, PreRequestEventArgs e)

    {

        if (!String.IsNullOrEmpty(this.proxy))

        {

            e.WebTest.Proxy = this.proxy;

        }

        else

        {

            e.WebTest.Proxy = null;

        }

    }

}

Importance of specifying the right FileAccess option.

When debugging an application that accepts a filename as input, I came across the following exception
System.UnauthorizedAccessException: Access to the path 'C:\test.txt' is denied.

The line at which it failed was
FileStream fs = new FileStream(path, FileMode.Open);

I checked my access permissions on the file and everything was perfect. Except that I noticed that this file test.txt was marked Read Only. When I changed the above line to include the FileAccess option
FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read);
the code executed successfully!

So moral of this is that wherever possible we should specify the FileAccess option for file operations.

Using .NET AutomationElement classes from Console Application.

When using .NET 3.0 UIAutomation classes in a console application, I was surprised when a call to AutomationElement.FindFirst returned a null value. The same method when called from a .NET WinForm application returned a valid AutomationElement.

If you encounter a similar issue, please bear in mind that UI Automation APIs only work from a STA thread. So if you ever want to get UIAutomation run from the console application, remember to decorate your Main function with the STAThreadAttribute.

Ref: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1706155&SiteID=1

SQL Server Reporting Services support for Firefox.

For a list of Reporting services features not supported in Firefox, check out http://msdn.microsoft.com/en-us/library/ms156511.aspx

Reading a INI file

A win32 function named GetPrivateProfile can be used to read an INI file.

Ref: http://msdn.microsoft.com/en-us/library/ms724353(VS.85).aspx

.NET Colors visual reference

I needed to visualize the different colors available for setting the BackColor and ForeColor properties of some controls. In my search, I came across this page. Thanks to the person putting this up for use of everybody. :)

Better Together Microsoft Project Plan + TFS.

Today, I learnt how to synchronize my MPP with TFS work items. I must confess that it is a delightful feature for any project manager!

You can access this productive feature from the Team menu of a MPP. Select Get Work Items and have your mpp populated with work items created in TFS. If you are not too happy with the field mappings, Microsoft provides a nice command line utility named TFSFieldMapping which resides in "%ProgramFiles%\Microsoft Visual Studio 9.0\Common7\IDE". Follow the steps given here and here to customize the mappings for your project.

Simple :)

Updating TFS Warehouse manually.

To update TFSWarehouse manually, you must access the machine hosting TFS and browse to
http://localhost:8080/Warehouse/v1.0/warehousecontroller.asmx

Then invoke the Run operation. This operation executes asynchronously. To see if the warehouse update is complete, invoke the operation GetWarehouseStatus. Simple!

Improving Website performance

While studying performance considerations for website design, I came across this site

http://developer.yahoo.com/performance/

that has good tips for developers to improve website performance.

Mobile Browsers

While researching on usage of Mobile Browsers, came across two interesting sites
1. Mobile browser statistics
2. Mobility checker

Logging and Performance.

What is the impact of logging on performance? If my ASP.NET website logs information about requests to a log file, the log file will turn out to be a resource that will get concurrently accessed. A lock placed on the log file will significantly impact ability of the server to concurrently process requests. Logging frameworks like log4net handle this very smartly.

Frequently used TFS commands

Some of the TFS commands I use frequently include
1. tfsdeleteproject
2. witimport
3. witexport
4. tf workspace
5. tf status: To check who has checked out a file Default.aspx, run the command
tf status Default.aspx /user:*

SqlBulkCopy vs SSIS Bulk Insert

I recently wrote a program to move data from a CSV file into a SQL Server table using SqlBulkCopy.
The CSV file had data like
0,0,35.6,-122.5,1,
0,1,35.61188,-122.4438,1,
0,2,35.61979,-122.4062,1,
.
.

Below are some of my observations.

Trailing Delimiter Handling
With SqlBulkCopy, the data was successfully moved to SQL Server.

But when I used the SSIS Bulk Insert Task, the following error was reported

Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 5
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 2, column 5
.
.

The fix for this was to remove the trailing ',' character at the end of each line in the CSV file as shown below and then perform SSIS Bulk Insert.
0,0,35.6,-122.5,1
0,1,35.61188,-122.4438,1
0,2,35.61979,-122.4062,1
.
.

Speed
I did not observe any significant difference in the data load speed of SqlBulkCopy & SSIS Bulk Insert task. I will post performance numbers at some later time.

Scheduling planned shutdown of Windows 2003 server

To perform a planned shutdown of Windows 2003 server at a specified time, the following command can be used

at 3:00am shutdown /s /d p:1:1

Using VS 2008 Code Metrics tool for SharePoint projects.

I ran the Code Metrics tool on a VS 2008 project containing a set of SharePoint web parts. The tool failed to show code metrics for Projects that had a reference to Microsoft.SharePoint DLL. Given below is the message I received.

An error occurred while calculating code metrics for target file 'D:\MyFolder\Projects\ConsoleApplication1\bin\Debug\ConsoleApplication1.exe' in project ConsoleApplication1. The following error was encountered while reading module 'Microsoft.SharePoint': Security attribute type does not have a default constructor: Microsoft.SharePoint.Security.SharePointPermissionAttribute, Microsoft.SharePoint.Security, Version=12.0.0.0, Culture=neutral, Publickeytoken=71e9bce111e9429c.

I found the same issue reported on Microsoft connect site, but it had been closed since Microsoft was unable to reproduce the issue. So I provided the steps to reproduce the issue and got back to my work.In the process, I created a dummy console application with a reference to Microsoft.SharePoint. After submitting the repro steps, I had a good look at the error message and thought: "What if I include a reference to the Microsoft.SharePoint.Security namespace? Will that help? Let me try..." I did that and ran the Code metric tool. Pronto, I had the code metrics for the SharePoint project in the Code Metric Results section!!! Doing something for the community somewhere down the line helps you, isn't it? :)

So if you run into a similar issue when using the Code Metrics tool, just add a reference to "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\ISAPI\Microsoft.SharePoint.Security.dll" and re-calculate the code metrics.

Troubleshooting a full transaction log.

While trying to create a new TFS project, I was running into issues creating the SharePoint site. I noticed that the SQL Server log had the following message

"The transaction log for the database 'WSS_Content' is full. To find out why the space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases".

So I ran the following query

select [name], [log_reuse_wait], [log_reuse_wait_desc] from sys.databases

and noticed that log_reuse_wait_desc had the value set to LOG_BACKUP.

This meant that a Log backup operation is preventing the truncation of log files and consequently its reuse. So I explicitly performed a Transaction Log backup through Management studio. With that the error message disappeared and the transaction log file became reusable.

Ref:
Troubleshooting a Full Transaction Log (Error 9002)

How to: Back Up a Transaction Log (SQL Server Management Studio)

Controlling form layout for TFS Work Items

Here is a very good post on the following undocumented attributes in TFS for controlling work item form layouts
1. Name
2. MinimumSize
3. NumberFormat
4. MaxLength
5. Format

TFS Backup: Remember the Agent

I scheduled a maintenance job to backup the TFS databases following the steps published on msdn. When I executed the plan, I noticed the following error in the SQL Server logs.

BackupDiskFile::CreateMedia: Backup device 'D:\Backups\WSS_Content_backup_200801111921.bak' failed to create. Operating system error 5(Access is denied.)

After some investigation, I noticed that the account under which SQL Server Agent was running did not have write permissions on the backup folder. After granting the appropriate permissions and re-executing the plan, backups got created and I heaved a sigh of relief!

The crawler could not communicate with the server

Recently, I saw the following error message logged by Office SharePoint Server Search service.
"The crawler could not communicate with the server"

The Search service had been working perfecly just a few days back. So this error, out of the blue, did surprise me. A bit of calm investigation revealed that the password associated with the service account had expired. Changing the password and restarting the service fixed this issue!

'Exec' task needs a command to execute.

After editing the post-build event of my project in VS 2005, the following error came up
"'Exec' task needs a command to execute". I realized that I had a new-line character still present in the post-build event. After clearing that I rebuilt the project successfully.

Deleting build history in TFS

If we delete the builds from Build Explorer, the same is not reflected in the drop down list of of the Bug work item. Benjamin shows us how to delete build history from TFS in this article.