File Transfer using sockets and multiple clients

I have a large application written using .Net remoting for file transfer. This was borking in some circumstances with the sockets being forcibly closed - I wasn't using sockets directly, but a .Net remoting call with byte arrays (I wasn't sending the whole file in one transfer, I was splitting it up).

So, I decided to change the actual file transfer part to use sockets.

As a proof of concept, to see if I got the principles right, I have written a simple console client and a server.

I am using ASynch recieves but synchronous writes - I have tried with both being ASync but same reseult, and keeping it synchronous made debugging easier.

What the apps do (code below) is the server sits and waits for files to be transfered, and it stores them in a directory at a given name.

When enter is pressed, the server then reads the files recieved and sends them back to the clients who store them under a different name. I wanted to test file transfer both ways.

Using one instance of the client application, all is well - the server recieves it and then sends it back to the client. All is well. Yes, the client throws an exception when you terminate the server - but that is fine - I know that the socket was forcibly closed...I can deal with tidying upu the code when it is working.

However, when I create 2 instances of the client code (not forgetting to modify the code slightly to read a different file to send, and also to store the received file under a different name) - the server receives both files from the clients, sends the first one back just fine, and then a few segments into the second file it throws with a "non blocking socket operation could not be completed immediatly" - which is odd because nothing is blocking, and the recieves are async - and the sends are actually blocking!

Any suggestions please as to what I am doing wrong - no doubt it is something stupid, but still...

The aim of the final code is to be able to have n clients contact the server and send files to it, and also, at random intervals have the server send 1 or more files back to some/all of the clients.

Cheers folks!

Server code

    using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Sockets;
using System.IO;

namespace SocketServer
{

    class ConnectionInfo
    {
        public Socket Socket;
        public byte[] Buffer;
        public int client;        
    }



    class Program
    {

        static int chunkSize = 16 * 1024;
        static int chucksizeWithoutHeaderData = chunkSize - 8;
        static List<ConnectionInfo> list = new List<ConnectionInfo>();

        static Socket serverSocket;

        static int nClient = 0;


        static void AcceptCallback(IAsyncResult result)
        {


            ConnectionInfo info = new ConnectionInfo();
            info.Socket = serverSocket.EndAccept(result);

            info.Buffer = new byte[chunkSize];


            Console.WriteLine("Client connected");
            nClient++;
            info.client = nClient;

            list.Add(info);

            info.Socket.BeginReceive(info.Buffer,0,info.Buffer.Length,SocketFlags.None, new AsyncCallback(ReceiveCallBack), info);

            serverSocket.BeginAccept(new AsyncCallback(AcceptCallback),null);

        }

        static void ReceiveCallBack(IAsyncResult result)
        {
            ConnectionInfo info = result.AsyncState as ConnectionInfo;
            try
            {

                Int32 nSegmentNumber = BitConverter.ToInt32(info.Buffer,0);
                Int32 nMaxSegment = BitConverter.ToInt32(info.Buffer,4);

                string strFileName = string.Format(@"c:\temp\from-client-{0}.dat",info.client);

                int bySize = info.Socket.EndReceive(result);
                using (FileStream fs = new FileStream(strFileName, FileMode.OpenOrCreate))
                {
                    Console.WriteLine("Received segment {0} of {1} from client {2}", nSegmentNumber, nMaxSegment, info.client);
                    fs.Position = fs.Length;
                    fs.Write(info.Buffer, 8, bySize-8);

                    if (nSegmentNumber >= nMaxSegment)
                    {
                        Console.WriteLine("Completed receipt from client {0}", info.client);
                    }

                }

                info.Socket.BeginReceive(info.Buffer, 0, info.Buffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallBack), info);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }



        static void Main(string[] args)
        {
            try
            {

                Console.WriteLine("Server");

                IPAddress address = IPAddress.Parse("127.0.0.1"); //The IP address of the server
                IPEndPoint myEndPoint = new IPEndPoint(address, 6503);
                serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);

                serverSocket.Bind(myEndPoint);

                serverSocket.Listen(1000);

                for (int n = 0; n < 10; ++n)
                {
                    serverSocket.BeginAccept(new AsyncCallback(AcceptCallback), null);
                }

                Console.WriteLine("Server now waiting");
                Console.ReadLine();

                foreach (ConnectionInfo info in list)
                {


                    string strFileName = string.Format(@"c:\temp\from-client-{0}.dat", info.client);

                    using (FileStream fs = new FileStream(strFileName, FileMode.Open, FileAccess.Read, FileShare.None))
                    {
                        int nMaxChunk = 0;
                        int nCurrentChunk = 0;

                        nMaxChunk = (int)(fs.Length / chucksizeWithoutHeaderData);
                        if ((nMaxChunk * chucksizeWithoutHeaderData) < fs.Length)
                        {
                            ++nMaxChunk;
                        }


                        using (BinaryReader br = new BinaryReader(fs))
    开发者_开发技巧                    {
                            byte[] byBuffer;
                            Int64 nAmount = 0;
                            byte[] byMaxChunk = BitConverter.GetBytes(nMaxChunk);
                            while (fs.Length > nAmount)
                            {
                                ++nCurrentChunk;

                                byte[] byCurrentChunk = BitConverter.GetBytes(nCurrentChunk);

                                byBuffer = br.ReadBytes(chucksizeWithoutHeaderData);


                                Console.WriteLine("Sending {0}bytes, chunk {1} of {2} to client {3}", byBuffer.Length,nCurrentChunk,nMaxChunk, info.client);

                                byte [] byTransmitBuffer = new byte[byBuffer.Length + 8];
                                Array.Copy(byCurrentChunk, byTransmitBuffer, 4);
                                Array.Copy(byMaxChunk, 0,byTransmitBuffer, 4, 4);
                                Array.Copy(byBuffer, 0, byTransmitBuffer, 8, byBuffer.Length);

                                info.Socket.Send(byTransmitBuffer);
                                nAmount += byBuffer.Length;
                            }
                        }
                    }
                }


                Console.WriteLine("Press enter to end server");
                Console.ReadLine();



            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
                Console.ReadLine();
            }
        } 

    }
}

Client code

    using System;
using System.Collections.Generic;
using System.Net.Sockets;
using System.IO;
using System.Threading;

namespace SocketClient
{
    class Program
    {
        static TcpClient socket = new TcpClient();
        static int chunkSize = 16 * 1024;
        static int chucksizeWithoutHeaderData = chunkSize - 8;
        static byte[] byReceiveBuffer = new byte[chunkSize];
        static void ReceiveCallBack(IAsyncResult result)
        {
            Socket socket = result.AsyncState as Socket;
            try
            {
                int bySize = socket.EndReceive(result);

                Console.WriteLine("Recieved bytes {0}", bySize);

                if (bySize != 0)
                {





                    Int32 nSegmentNumber = BitConverter.ToInt32(byReceiveBuffer, 0);
                    Int32 nMaxSegment = BitConverter.ToInt32(byReceiveBuffer, 4);

                    Console.WriteLine("Received segment {0} of {1}", nSegmentNumber, nMaxSegment);

                    string strFileName = string.Format(@"c:\temp\client-from-server.dat");
                    using (FileStream fs = new FileStream(strFileName, FileMode.OpenOrCreate))
                    {
                        fs.Position = fs.Length;
                        fs.Write(byReceiveBuffer, 8, bySize-8);
                    }

                    if (nSegmentNumber >= nMaxSegment)
                    {
                        Console.WriteLine("all done");
                    }
                }


                socket.BeginReceive(byReceiveBuffer, 0, byReceiveBuffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallBack), socket);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }



        static void Main(string[] args)
        {
            Console.WriteLine("Press enter to go");
            Console.ReadLine();


            socket.Connect("127.0.0.1", 6503);

            Console.WriteLine("Client");
            Console.ReadLine();

            byte[] byBuffer;




            socket.Client.BeginReceive(byReceiveBuffer, 0, byReceiveBuffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallBack), socket.Client);


            using (FileStream fs = new FileStream(@"c:\temp\filetosend.jpg", FileMode.Open, FileAccess.Read, FileShare.None))
             {



                 using (BinaryReader br = new BinaryReader(fs))
                 {

                     int nMaxChunk = 0;
                     int nCurrentChunk = 0;

                     nMaxChunk = (int)(fs.Length / chucksizeWithoutHeaderData);
                     if ((nMaxChunk * chucksizeWithoutHeaderData) < fs.Length)
                     {
                         ++nMaxChunk;
                     }

                     byte[] byMaxChunk = BitConverter.GetBytes(nMaxChunk);

                        Int64 nAmount = 0;

                        while (fs.Length > nAmount)
                        {
                            ++nCurrentChunk;
                            byte[] byCurrentChunk = BitConverter.GetBytes(nCurrentChunk);

                            byBuffer = br.ReadBytes(chucksizeWithoutHeaderData);
                            Console.WriteLine("Sending {0}bytes, chunk {1} of {2}", byBuffer.Length, nCurrentChunk, nMaxChunk);

                            byte[] byTransmitBuffer = new byte[byBuffer.Length + 8];
                            Array.Copy(byCurrentChunk, byTransmitBuffer, 4);
                            Array.Copy(byMaxChunk, 0, byTransmitBuffer, 4, 4);
                            Array.Copy(byBuffer, 0, byTransmitBuffer, 8, byBuffer.Length);

                            socket.Client.Send(byTransmitBuffer);
                            nAmount += byBuffer.Length;
                        }


                 }
             }



            Console.WriteLine("done");


            Console.ReadLine();



        }
    }
}

---

Since you are modifyng the infrastructure, use an FTP library in C# and install some free FTP server ( ie FileZilla or anyone else ). You can easily use some FTP library as for example this one that is reliable ( i used it on production code ).

---

Socket programming can be tricky. If you send 100 bytes, it doesn't mean you will receive 100 bytes in your server. You can receive those 100 bytes in multiple packets, you have to add code to control that.

I said that because of this line in your server code:

fs.Write(info.Buffer, 8, bySize-8);

you are assuming you will receive at least 8 bits and it can be wrong. bySize can be smaller than your chunk size (and it can be zero if the connection has been closed by the client or <0 if there was an error.)

About your error, I tested your code and I could replicate your problem:

• start the server
• start the client and transfer the file
• exit the client pressing enter
• the server crashes

The server crashes because the socket is waiting for more data after all the file is transferred. And the client closed it.

I solved the problem closing the connection in the client after the file is sent:

socket.Client.Disconnect(false);

Then the server receives bySize=0 bytes, meaning that the connection was closed. In the server I replaced this:

int bySize = socket.EndReceive(result);

for this:

int bySize = 0;
try
{
    bySize = info.Socket.EndReceive(result);
}
catch (Exception ex)
{
    Console.WriteLine("Error from client {0}: {1}", info.client, ex.Message);
    return;
}

if (bySize <= 0)
    return;

Have a look here: http://msdn.microsoft.com/en-us/library/5w7b7x5f.aspx#Y240

and here: http://msdn.microsoft.com/en-us/library/fx6588te.aspx

EDIT: I forgot to mention this, you only need to call BeginAccept once. I removed the for statement.

// for (int n = 0; n < 10; ++n)
// {
    serverSocket.BeginAccept(new AsyncCallback(AcceptCallback), null);
// }

---

I think that I may have a solution - although I am not 100% confident...I will keep testing and report back if it fails.

Anyways - I have set the socket.SendBufferSize and RecieveBufferSize to 4 x the chunk size, and now all seems to be well.

The thing there (re buffer size) just put the problem off, as I thought it might.

However, I came across a snippet of code elsewhere, and putting these lines into the code fixed the issue.

  try
                            {
                                bySent = info.Socket.Send(byTransmitBuffer);
                            }
                            catch (SocketException ex)
                            {
                                Console.WriteLine("Only sent {0}, remaining = {1}", bySent,fs.Length -nAmount);
                                if (ex.SocketErrorCode == SocketError.WouldBlock ||
                                  ex.SocketErrorCode == SocketError.IOPending ||
                                  ex.SocketErrorCode == SocketError.NoBufferSpaceAvailable)
                                {
                                    // socket buffer is probably full, wait and try again
                                    Thread.Sleep(30);
                                }
                                else
                                    throw ex;  // any serious error occurr
                            }