How faster is sending/receiving values by UNIX domain socket than starting new processes when executing commands

linuxgolang

For example when some procedures are written in different languages and they are repeatedly executed as different commands, overhead such as memory allocation occurs when creating a process. One of solutions to avoid this is to keep the process running and pass input/output value with interprocess communication. In this article I make a command in Go and benchmark to see how much the throughput differs.

For speedy interprocess communication on UNIX, besides shared memory, pipes and UNIX domain sockets are used. For example, newrelic-lambda-extension receives application events from pipe and send it. Since both input and output are needed this time, I use UNIX domain socket instead of pipe which can communicate only in one direction. The entire codes are on GitHub.

About newrelic-lambda-extension and how it works telemetry without CloudWatch Logs - sambaiz-net

Create a command

Create a command with cobra.

$ go install github.com/spf13/[email protected]
$ cobra-cli init

If –server flag is passed, wait for requests by UNIX domain socket, otherwise echo the value passed as –value.

func handler(in []byte) []byte {
	return in
}

var rootCmd = &cobra.Command{
	Use: "start-process-vs-unix-domain-socket",
	RunE: func(cmd *cobra.Command, args []string) error {
		daemon, err := cmd.Flags().GetBool("server")
		if err != nil {
			return err
		}
		if daemon {
			socketName, err := cmd.Flags().GetString("socket")
			if err != nil {
				return err
			}
			if err := listenSocket(socketName); err != nil {
				return err
			}
		} else {
			value, err := cmd.Flags().GetString("value")
			if err != nil {
				return err
			}
			fmt.Println(string(handler([]byte(value))))
		}
		return nil
	},
}

func Execute() {
	err := rootCmd.Execute()
	if err != nil {
		os.Exit(1)
	}
}

func init() {
	rootCmd.Flags().Bool("server", false, "Wait for the value with socket")
	rootCmd.Flags().String("socket", "rpos.sock", "Socket name to be listen by the server")
	rootCmd.Flags().String("value", "", "Value to be processed")
}

Communication on UNIX domain socket

On the server side, wait for requests with net.Listen(“unix”, socketName) as follows and Accept() then read and write.

func listenSocket(socketName string) error {
	l, err := net.Listen("unix", socketName)
	if err != nil {
		return err
	}
	defer l.Close()

	log.Printf("listen on socket %s", socketName)

	connCh := make(chan net.Conn)
	errCh := make(chan error)
	go func() {
		for {
			conn, err := l.Accept()
			if err != nil {
				errCh <- err
				return
			}
			connCh <- conn
		}
	}()

	sigs := make(chan os.Signal, 1)
	signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)

	for {
		select {
		case conn := <-connCh:
			log.Print("connected")
			go func() {
				defer conn.Close()
				buf := make([]byte, 512)
				for {
					if _, err := conn.Read(buf); err != nil {
						log.Print(err)
						return
					}
					if _, err := conn.Write(handler(buf)); err != nil {
						log.Print(err)
						return
					}
				}
			}()
		case err := <-errCh:
			return err
		case <-sigs:
			return nil
		}
	}
}

On the client side, connect with net.Dial(“unix”, socketName) then read and write.

conn, err := net.Dial("unix", "test.sock")
if err != nil {
    b.Fatal(err)
}

conn.Write([]byte(value))
if _, err := conn.Read(buf); err != nil {
    b.Fatal(err)
}
_ = conn.Close()

Benchmarks

Benchmarked with -benchtime (100|1000|10000)x.

BenchmarkProcess-4                       	     100	   4433338 ns/op
BenchmarkSocketDialEveryTime-4           	     100	  10198513 ns/op
BenchmarkSocketDialOnce-4                	     100	  10109449 ns/op
BenchmarkSocketAlreadyProcessStarted-4   	     100	    101352 ns/op

BenchmarkProcess-4                       	    1000	   5761057 ns/op
BenchmarkSocketDialEveryTime-4           	    1000	   1105874 ns/op
BenchmarkSocketDialOnce-4                	    1000	   1026624 ns/op
BenchmarkSocketAlreadyProcessStarted-4   	    1000	     14375 ns/op

BenchmarkProcess-4                       	   10000	   5112125 ns/op
BenchmarkSocketDialEveryTime-4           	   10000	    205253 ns/op
BenchmarkSocketDialOnce-4                	   10000	    113940 ns/op
BenchmarkSocketAlreadyProcessStarted-4   	   10000	     13312 ns/op

If the number of run times is small, the cost of starting the server is higher than the cost of creating processes, so BenchmarkProcess which creates a process each time is faster than BenchmarkSocketDialEveryTime and BenchmarkSocketDialOnce that use UNIX domain socket but it reverses as the number increses. In fact BenchmarkProcess took about 50 seconds, but BenchmarkSocketDialOnce took only in 1 second on 10000 times!

To see the result of BenchmarkSocketAlreadyProcessStarted which is except for server startup, BenchmarkSocketDialEveryTime which connects the server each time, and BenchmarkProcess which reuse the connection, the cost of the connection isn’t as high as starting the server, but it’s almost doubled on 10000 times, so it’s better to reuse it if possible.