How do I get an initial Ignite instance on restarting a website in IIS?
I am using Apache Ignite.NET EntityFramework second level caching on my ASP.NET MVC 5 website. Everything works as expected on my development machine, but things get complicated when I try to publish the website using Web Deploy on production IIS.
I always get class org.apache.ignite.IgniteException: Ignite instance with this name has already been started: GridName
when I publish or restart a site in IIS. The error is pretty clear, Ignite is already running.
The only way to avoid this error is to restart the application pool on the website. However, I don't want to do this every time I publish a website in production (which happens a few times a week).
I tried to stop the Ignite instance in global.asax Dispose () or Application_End (), but the thing is, the AppDomain takes many seconds to stop. Therefore Ignite has time to try to start itself before it is stopped and throws the error mentioned above.
I also tried calling Ignition.TryGetIgnite()
to fetch the executable instance instead of trying to start it, but it always returns null. Looking at the source code for this function in the Apache Ignite Github repository, I see that the Ignition object simply stores a static list of nodes in memory and performs operations on that list. Since the AppDomain restarted at this point, the list is empty, but the Ignite node is still running in the JVM.
Is there a way to get the Ignite instance or stop it reliably so I don't have to restart the application pool every time?
source to share
Not exactly what I was looking for, but I have solved the problem for now.
What I am doing is in Application_End
, I call Ignition.StopAll()
, as suggested by Paul, but since it Application_End
sometimes takes a long time to call, a new AppDomain loaded for a website has time to start and receive requests before Ignite stops at another AppDomain.
To get around this, I start Ignite using the following code (which could be improved):
//Try to get already launched Ignite Instance
IIgnite ignite = Ignition.TryGetIgnite(igniteName);
while (ignite == null)
{
try
{
//Try to start Ignite
ignite = Ignition.Start(igniteConfig);
}
catch (Exception) //If failing to start Ignite, wait a bit for the previous AppDomain to stop the Ignite running instance...
{
HttpRequest request = null;
try
{
request = HttpContext.Current.Request;
}
catch { }
//Check if there is a request coming from the same machine, if yes, cancel it.
if (request == null || !(request.IsLocal || request.UserHostName.EndsWith("myhostname.com"))
Thread.Sleep(1000);
else
throw new HttpException(500, "Server error. Server rebooting.");
}
}
//Use ignite variable here...
Please note that in the code above, I am adding an additional check (which everyone might not need) to check if there is a current request coming from the same computer. If you want to know why, read below, otherwise just grab the code and remove the last part if else
and just save Thread.Sleep()
.
If the request is sent from the same computer, the HttpException is thrown to cancel the request, otherwise it persists. The reason I am putting this code is because if I publish the website to IIS, I don't want users to see the error, but I don't mind if they wait a bit for the previous AppDomain to stop. But if I get a request from the same machine (or a website), I would like to request interrupted because it probably comes from the code of the old AppDomain and therefore delays Application_End
and Ignition.StopAll()
from the old AppDomain.
Hopefully some future development in Apache Ignite.NET can make closing an instance easier and more reliable than this workaround.
source to share
This is a known issue: The AppDomain has stopped but the JVM continues to run (because the process is not stopped), so the Java portion of the Ignite nodes still exists.
A workaround is to stop all Ignite nodes with Ignition.StopAll
an Application_End
event like this in Global.asax.cs
:
protected void Application_Start()
{
...
using (new Mutex(true, "ignite_" + Process.GetCurrentProcess().Id))
{
var ignite = Ignition.TryGetIgnite() ?? Ignition.Start();
}
}
protected void Application_End()
{
using (new Mutex(true, "ignite_" + Process.GetCurrentProcess().Id))
{
Ignition.StopAll(true);
}
}
The mutex is needed here because the stop of the old domain overlaps with the start of the new domain. The process ID is included in the mutex name to ensure that the entire process is locked, but not the entire machine.
Another workaround , possibly more reliable and cleaner:
1) Stop all nodes in AppDomain.DomainUnload:
AppDomain.CurrentDomain.DomainUnload += (sender, args) => Ignition.StopAll(true);
2) Use a different one each time IgniteConfiguration.GridName
:
Ignition.Start(new IgniteConfiguration { GridName = Guid.NewGuid().ToString() });
This way, existing nodes won't prevent you from starting a new one. Eventually the old nodes will be stopped and cleared from memory.
Documentation: https://apacheignite-net.readme.io/docs/deployment#section-aspnet-deployment
source to share