Strange URL bypassing routing, parentheses in parentheses, in parentheses, two closing parentheses
I stumbled upon some strange behavior on a website at work today. Our SEO consultant wanted some interesting links removed from the Google index, which would seem to be a straight forward task. But it turned out to be very difficult.
The website was .net MVC 5.2.3. We looked at routing, our own libraries, etc. Nothing strange. After a while, we gave up and tried to simply redirect the request to these URLs by setting a rule in web.config. Turns off these URLs: s unmatchable! Somehow, given the right conditions, the critical part of the url seems to be avoiding rule matching as well as routing later in the MVC application.
We've narrowed down mystical URL: s in a format (T(anything))
where T can be any capital letter, and anything can be eh, whatever. It is placed at the beginning of the url as if it were a directory. In regex:\([A-Z]\([a-zA-Z0-9]*\)\)
I tested and found the same behavior:
- .net MVC5 sites
- .net sites MVC3
- .net Web Forms
- http://asp.net
- /qaru.site / ...
Some examples from stackoverflow.com:
- Bypasses routing: stackoverflow
-
The routes are normal (404): stackoverflow
-
Bypasses routing: https://stackoverflow.com/(G(hello))/users/1049710/jonas-%C3%84ppelgran
- Routes normal (404): https://stackoverflow.com/gandhello/users/1049710/jonas-Äppelgran
It doesn't affect the entire network, so it shouldn't be a browser or HTTP issue. Some examples:
- Routes normal (404): http://php.net/(T(testing))/downloads
- Normal routes (404): https://www.iana.org/(T(testing))/domains/reserved
Can anyone please explain what's going on?
And what can I do to prevent these urls: s from bypassing routing?
source to share
Apparently this is a feature called "cookieless session" in ASP.NET. See the "Cookieless SessionIDs" section here in the MSDN docs.
The basic idea is that instead of storing the session ID (if session state is enabled) in a cookie, it is now embedded in the URL.
We (Stack Overflow) have completely disabled the session state (by setting the mode sessionState
to off
). As far as I know, the end result is that whenever one of the URLs that match the Session ID format is used, that information is simply discarded.
None of the links leading to us on Google include this, which makes me think your site might be configured to actually create session IDs in URLs? If you do not disable this feature, you are unlikely to be able to do this. Although, see "Regenerating Past Session IDs" on the MSDN page I linked to above for how to at least prevent accidental session sharing if you haven't already.
source to share