What dangers can the update of .Net 4.6.1 to .Net 4.6.2 hide?

In this article, I would like to share a solution of unexpected problem occurred in one of the projects I am currently working on.

What dangers can the update of .Net 4.6.1 to .Net 4.6.2 hide? I thought that to avoid serious issues during the update to a minor version, it was enough to read the release notes. However, it turned out that Microsoft can make interesting and entertaining changes beyond release notes that can divert you on soulful summer nights and days off.

In the Beginning

A scheduled update of .Net. Why? Just because! We can dispute about “If it works, don’t touch it.” for a long time, but I think that regular updates of the stack components are required for active projects. Otherwise, at some point, updates will turn into a big pain after few dozens of versions. Consequently, it will be easier to freeze up the old version than to try updating and solving problems that community tried to solve a few years ago and nobody remembers the details.

Generally speaking, a scheduled update should have been for version 4.7. But since Microsoft has been redesigning its partnership system for over half a year and did not provide an adequate way to prolong it under our conditions, we still don’t have VS 2017. That is why we decided to update to version 4.6.2.

So said, so done. We updated, deployed on the testing environment, tested and deployed to the production environment. Everything was fine.

Problems

System.Web.HttpException (0x80004005): The request queue limit of the session is exceeded.

Emm, what? The daily analysis of the most frequent errors revealed 600+ such errors on the lightly loaded server. OK, let’s start digging.
At the same time, the support for http/2 was enabled on mobile devices and the old asmx service was found in the stack. So, the trace was doubled a bit. We began to suspect that it was an issue of frequent queries that overloaded the queue.

[expand title =”asmx”]

Yes, we still use webApi2, there are still a number of asmx services that are migrated gradually due to the project size.[/expand]

We began to debug this case. Since all mobile services worked in the read-only mode with the session, we switched the session to the corresponding mode.

Global.asax:

if (<Query to the crushing service>)
{
    Context.SetSessionStateBehavior(SessionStateBehavior.ReadOnly);
}

Next, we updated the environment. The problem with mobile devices disappeared. But, they occurred with another asmsx service that was used from the web.
This fact brought us to the conclusion that the initial assumption was wrong.

Help us, Google!

Let’s take a look at the solutions found with Google. One of the variants was to increase requestQueueLimit. We tried it, but with no success. Moreover, the load does not prove that this limit can be broken through somehow. Another advice was to increase resources if you have issues with queue without touching a queue. This variant wasn’t good for us. But there was one more variant — source. I am really grateful to Microsoft for opening source code. Otherwise, many problems would be solved empirically which is ten times longer and harder or would remain unsolved at all.

Input data:

System.Web.HttpException (0x80004005): The request queue limit of the session is exceeded.
at System.Web.SessionState.SessionStateModule.QueueRef()
at System.Web.SessionState.SessionStateModule.PollLockedSession()
at System.Web.SessionState.SessionStateModule.GetSessionStateItem()
at System.Web.SessionState.SessionStateModule.BeginAcquireState(Object source, EventArgs e, AsyncCallback cb, Object extraData)

Let’s check the whole chain and get to the final method:

private void QueueRef() {
    if (!IsRequestQueueEnabled || _rqId == null) { 
        return;
    }

    //
    // Check the limit
    int count = 0;
    s_queuedRequestsNumPerSession.TryGetValue(_rqId, out count);

    if (count >= AppSettings.RequestQueueLimitPerSession) {
        throw new HttpException(SR.GetString(SR.Request_Queue_Limit_Per_Session_Exceeded));
    }

    //
    // Add ref
    s_queuedRequestsNumPerSession.AddOrUpdate(_rqId, 1, (key, value) => value + 1);
}

Hmm, what is the RequestQueueLimitPerSession setting for (it is mentioned not only in .Net 4.7 but is also extended to the fixes up to .net 3.5)?

Let’s switch to AppSettings.cs for investigating the setting. Bingo!

If you have .net 4.6.3 (which appears to be a system name for 4.6.2) or older, the queue length for each user is limited to 50 queries that are reached quite easily in certain use cases. We increased the limit, uploaded, tested – happy end.

I hope that this article will help someone who may face the same problem since it is quite hard to find a solution to this problem on the web.

P.S. It is strange that such changes are made bypassing the release notes. It seems that the changes were distributed with the updates and fixes from 4.7. From my point of view, it is an obvious breaking change that partly dumped the application for some time. Anyway, I would like to see such changes more explicitly.