well, im not sure about IrDA, but with a regular socket class you can specify the port in the endpoint that the socket listens to, like this:
IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
IPAddress ipAddress = ipHostInfo.AddressList[0];
IPEndPoint localEndPoint = new IPEndPoint(ipAddress, port);
Socket listener = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp );
listener.Bind(localEndPoint);
listener.Listen(number_of_backlogs);
then you can do a listener.Accept() and listener.Recieve(),
Theres a really good tutorial on sockets in the msdn.
Jimmy