Multiple instances

I am trying to perform parallel processing by lauching a console application program2 that does the work stuff. It is launched by program1 which knows how many instances to launch.

At some point the program can't launch more instances. Even if you increase the instancesmount, it only launches to a limit. In this case only 92. if I set the limit to 100 or 200 it still only launches 92 on the server.

I am writing the program in c# and it runs in windows server 2008.

Here is the code:

for (int instanceCount = 0; instanceCount < InstancesAmount; instanceCount++)
{    
    using (System.Diagnostics.Process myProcess = new System.Diagnostics.Process())
    {
        if (hiddeConsoleWindow)
        {
            myProcess.StartInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden; 
        }

        myProcess.StartInfo.FileName = ExecutablePathProgram2;

        System.Security.SecureString password = new System.Security.SecureString();
        foreach (char c in ConfigurationSettingsManager.ProcessStartPassword.ToCharArray())
        {
            password.AppendChar(c);
        }

        myProcess.Sta开发者_开发问答rtInfo.UserName = ConfigurationSettingsManager.ProcessStartUserName;
        myProcess.StartInfo.Password = password;
        myProcess.StartInfo.Domain = ConfigurationSettingsManager.ProcessStartDomain;
        myProcess.StartInfo.UseShellExecute = false;

        myProcess.Start();
    }
}

I have been looking if there is max instances to launch but it always says that it is as many as the OS supports.

I also checked if there is a max instances per session or per user but couldn't find anything that describes something like that or I did miss it.

---

To quote Raymond Chen's blog: "If you have to ask about various operating system limits, you're probably doing something wrong".

There's a limit to how much work can actually get done by the computer even with that many processes. You will be better served by determining the number of processors in the system and choosing that many concurrent tasks to execute. Your "program1" could then launch the process and use the StartInfo to watch for when the process ends (and capture any error output in the meantime by redirecting the output and error streams and logging them as needed. Once a process completes, then you should launch the next one in the queue.

When you launch that many processes, the system will be thrashing trying to switch context between 100 processes and won't get much of anything done.

You may be running into memory limits depending on how much memory your child processes allocate. You'll have a bunch of processes starting up and taking up chunks of memory but doing nothing until their turn at the processor comes around. If it can't allocate the memory, it will probably choke and kill the process (depending on how error handling is done).

---

That is strange, as there is no hard limit by default. But of course, that depends on what is the launched process doing (memory consumption, handle allocation, files, etc.). Fo example, I have tested it with "notepad.exe" on my machine and I get 150 notepad.exe running if I specify 150 instances.

You can check here for a very interesting discussion on process limits: Pushing the Limits of Windows: Processes and Threads.

---

First, I definitely agree with @Garo Yeriazarian. But to be thorough I'd recommend checking out this blog post: http://xentelworker.blogspot.com/2005/10/i-open-100-explorer-windows-and-my.html