Debugger

Installation

go get github.com/go-delve/delve/cmd/dlv

Set breakpoint

import "runtime"

func someFunction() {
    // ... some code ...

    runtime.Breakpoint() // This will act as your breakpoint

    // ... more code ...
}

Run

dlv debug yourprogram.go

or

dlv test

Commands

• continue or c: Continue execution until the next breakpoint.
• step or s: Execute the next line of code.
• next or n: Continue to the next line in the current function.
• list or ls: Show source code.
• print or p: Evaluate and print an expression.
• exit or quit: Exit the debugger.

For example, print an variable

(dlv) p string(char)
"a"

Integrate with air and VScode

.air.toml:

# Config file for air
[build]
  # -gcflags='all=-N -l' is crucial for dlv showing values of variables correctly in VARIABLE panel on VScode
  cmd = "go build -gcflags='all=-N -l' -o ./tmp/main ."
  bin = "tmp/main"
  full_bin = "dlv exec ./tmp/main --headless=true --listen=:2345 --api-version=2 --accept-multiclient --log --continue tmp/main --"
  include_ext = ["go", "tpl", "tmpl", "html"]
  exclude_dir = ["assets", "tmp", "vendor"]
  exclude_file = []
  follow_symlink = true
  root = "."
  tmp_dir = "tmp"
  build_delay = 200
  kill_delay = 500
  include_dir = []

[log]
  color = true
  timestamp = false

[notify]
  interval = 500
  medium = "log"

[live]
  full = true
  live_delay = 1000
  delay = 1000

[cmd]
  hook = "sh -c 'go generate ./...'"
  watch = false
  shell = "/bin/sh"

docker-compose.yml

ports:
  - "3000:3000"
  - "2345:2345"

Dockerfile:

FROM golang:1.22-alpine

# Install air for live reloading
RUN go install github.com/air-verse/air@latest
RUN go install github.com/go-delve/delve/cmd/dlv@latest

# Set the Current Working Directory inside the container
WORKDIR /app

# Copy go.mod and go.sum files
COPY go.mod go.sum ./

# Download all dependencies. Dependencies will be cached if the go.mod and go.sum files are not changed
RUN go mod download

# Copy the source code into the container
COPY . .

# Command to run air for live reloading
CMD ["air"]

launch.json in VScode

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Attach to Delve",
            "type": "go",
            "request": "attach",
            "mode": "remote",
            "remotePath": "/app", // Path inside the container
            "port": 2345, // Delve listening port
            "host": "127.0.0.1", // Usually localhost unless you're connecting to a remote server
            "apiVersion": 2,
            "showLog": true,
            "trace": "verbose"
        }
    ]
}

After launching container, you can use lsof -i :2345 to check if dlv is running successfully

How to debug?

1. Make sure VScode connect to dlv successfully
   1. Open Debug panel on sidebar or pressing Cmd+Shift+D
   2. Open DEBUG CONSOLE on bottom menu
   3. Press F5 to connect to dlv, you shonldn’t see any error
2. Set the breakpoint at your desired line
   • You will see SetBreakPointsResponse in DEBUG CONSOLE
3. Then run the code to hit the breakpoint
   • You will see the code that stops at that line in highlight mode
4. You can check the value of the variable by hovering over it

redigo

github.com/garyburd/redigo/redis

Connect to redis server

One connection

conn, err := redis.Dial("tcp", ":6379")
if err != nil {
    panic(err)
}
defer conn.Close()

Connection pool

pool := &redis.Pool{
    MaxIdle:     300,               // Maximum number of idle connections in the pool.
    MaxActive:   5,                 // Maximum number of connections allocated by the pool at a given time.
    IdleTimeout: 20 * time.Second,  // Close connections after remaining idle for this duration. Applications should set the timeout to a value less than the server's timeout.
    Dial: func() (redis.Conn, error) {
        c, err := redis.Dial("tcp", "127.0.0.1:6379")
        if err != nil {
            return nil, err
        }
        return c, err
    },
    TestOnBorrow: func(c redis.Conn, t time.Time) error {
        _, err := c.Do("PING")
        return err
    },
}

// 檢查連線是否成功
_, err := pool.Dial()
if err != nil {
    return nil, err
}

Command

GET

val, err := redis.String(conn.Do("GET", "key"))

SET

_, err := conn.Do("SET", workerID, "value")

DEL

_, err := conn.Do("DEL", "key")

LLEN

count, err := redis.Int(conn.Do("LLEN", "list_key"))

LPOP

val, err := redis.Bytes(conn.Do("LPOP", "list_key"))

RPUSH

_, err := conn.Do("RPUSH", "list_key", "val")

HGETALL

res, err = redis.StringMap(conn.Do("HGETALL", "key"))

MHSET

var args []interface{}
args = append(args, "my_hash_key")
for field, val := range hash {
args = append(args, field, val)
}
_, err = conn.Do("HMSET", args...)

or

_, err = conn.Do("HMSET", redis.Args{}.Add("my_hash_key").AddFlat(hash)...)

HGET

val, err = redis.String(conn.Do("HGET", "key", "field"))

HSET

_, err = conn.Do("HSET", key, field, value)

KEYS (show keys with prefix)

vals, err := redis.Values(conn.Do("KEYS", "device:*"))

// 將第一個 key 取出來
_, err := redis.Scan(vals, &value)

其他

如果重覆用同一條 connection 做 del key 的動作可能會引發隨機出現的錯誤

use of closed network connection, short write

原因 : Concurrent writes are not supported

解決方法 : 使用 connection pool，要用的時候就拿一個新的 connection，不用的時候再放回去

Error dial tcp 127.0.0.1:6379: too many open files

使用 pool 要注意取出來的 connection 要釋放, 如果沒釋放有可能會出現此 error，因為系統資源的限制, 一個 process 不能超過 1024 個 socket

redisConn := redisPool.Get()
defer redisConn.Close()

goworker

介紹

Goworker 是 golang 的一套 job queue 的 package，它可以幫你達成這些事，它是使用 resque 的資料格式(resque 有固定的資料格式)， resque 是一套 Ruby 開發的 job queue，也有被其他的語言開發成該語言的版本，你可以把它當作是 golang 版的 resque， 最主要的好處是你後端語言可以用 php, ruby etc. 你想的語言寫，把它丟到 redis 裡的 job queue, 然候 goworker 再去拿， 並針對不同的 task 寫出不同對應的程式

Worker

package main

import (
    "fmt"
    "time"

    "github.com/benmanns/goworker"
)

func myFunc(queue string, args ...interface{}) error {
    fmt.Printf("From %s, %v\n", queue, args)
    return nil
}

func init() {
    goworker.Register("MyClass", myFunc)
}

func main() {
    if err := goworker.Work(); err != nil {
        fmt.Println("Error:", err)
    }
    fmt.Printf("Started on %v", time.Now().Format("2006-01-02 15:04:05"))
}

Run :

go run main.go -queues=MyClass

增加一筆 job 讓 worker 執行

你也可以用其他語言 insert 一筆 job

package main

import (
    "encoding/json"
    "log"
    "time"

    "github.com/garyburd/redigo/redis"
)

var redisPool redis.Pool

func init() {
    redisPool = redis.Pool{
        MaxIdle:     3,
        MaxActive:   0, // When zero, there is no limit on the number of connections in the pool.
        IdleTimeout: 30 * time.Second,
        Dial: func() (redis.Conn, error) {
            conn, err := redis.Dial("tcp", "127.0.0.1:6379")
            if err != nil {
                log.Fatal(err.Error())
            }
            return conn, err
        },
    }
}

func main() {
    redisConn := redisPool.Get()
    x := map[string]interface{}{
        "foo": []string{"a", "b"},
        "bar": "foo",
        "baz": 10.4,
    }
    resque := map[string]interface{}{
        "class": "MyClass",
        "args":  []interface{}{x},
    }
    b, _ := json.Marshal(resque)
    redisConn.Do("RPUSH", "resque:queue:MyClass", string(b[:]))
}

Run :

go run qq.go

結果 :

worker 就會 print 那筆 job 的資料

Dump (for debug) - spew

a := map[string]int64{}
a["A"] = 1
a["B"] = 2
debug.Dump(a)

執行結果 :

(map[string]int64) (len=2) {
 (string) (len=1) "A": (int64) 1,
 (string) (len=1) "B": (int64) 2
}

gorequest

agent := gorequest.New().CustomMethod("POST", "url").Timeout(30 * time.Second)
agent.Header = header               // 將 HEADER 以 map 型態傳進去
agent.Send(post_data)               // POST 需要, GET 不用, 傳入 String
resp, body, errs := agent.End()
if errs != nil {
    // Do something
}

if resp.StatusCode != 200 {
    // Do something
}

ORM

package main

import (
    "database/sql"
    "github.com/coopernurse/gorp"
    _ "github.com/go-sql-driver/mysql"
    "log"
)

func main() {
    // initialize the DbMap
    dbmap := initDb()
    defer dbmap.Db.Close()

    err := dbmap.Insert(&Tt{Name: "test"})
    checkErr(err, "Insert failed")
}

type Tt struct {
    Name string
}

func initDb() *gorp.DbMap {
    // connect to db using standard Go database/sql API
    // use whatever database/sql driver you wish
    db, err := sql.Open("mysql", "root:password@/go_test")
    checkErr(err, "sql.Open failed")
    dbmap := &gorp.DbMap{Db: db, Dialect: gorp.MySQLDialect{}}
    dbmap.AddTableWithName(Tt{}, "tt")
    return dbmap
}

func checkErr(err error, msg string) {
    if err != nil {
        log.Fatalln(msg, err)
    }
}

fasthttp

效能比原生的 http 好

import (
    "github.com/buaazp/fasthttprouter"
    "github.com/valyala/fasthttp"
)

router := fasthttprouter.New()
router.GET("/", Index)
fasthttp.ListenAndServe(":8010", router.Handler)

func Index(ctx *fasthttp.RequestCtx, ps fasthttprouter.Params) {
    // `Get` num params
    n := string(ctx.FormValue("num"))

    // 輸出
    fmt.Fprintf(ctx, "hello, %s!n", ps.ByName("name"))
}

帶入 net.Listener 的方式

l, err := net.Listen("tcp", ":"+strconv.Itoa(port))
if err != nil {
    fmt.Println("net.Listen error: %v", err)
    os.Exit(1)
}
router := fasthttprouter.New()
router.GET("/", Index)
err = fasthttp.Serve(l, router.Handler)
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}

go-logging

一套 log 的 package 特色如下 :

• 可以自由的設計 log format
• 可以自動地紀錄 call log 時當時是在什麼 package 及 func, 在 debug 時相當好用
• 可以設定當 log 達到什麼層級時另外紀錄下來
• 輸出不同 level 的 log 時有顏色

Example :

// 標準輸出的 log
logger = logging.MustGetLogger("example")
format := logging.MustStringFormatter(`%{color}%{time:15:04:05} %{shortpkg} %{shortfunc} [%{level}] %{color:reset} %{message}`)
backend := logging.NewLogBackend(os.Stderr, "", 0)
backendFormatter := logging.NewBackendFormatter(backend, format)

// 如果發生 Error 層級以上的 log 另外做處理
// 如果 buffer 有值就 post 到 slack
var buf bytes.Buffer
go func() {
    // 攔截 log 的內容送到 slack
    for {
        if buf.Len() > 0 {
            fmt.Println(buf.String())   // 這段可以改成 post 到 slack
            buf.Reset()                 // 將 buffer 清空
        }
        time.Sleep(1 * time.Second)     // 每秒檢查一次
    }
}()
// 將會進到 backend2 的 log 暫存到 buffer
backend2 := logging.NewLogBackend(&buf, "", 0)
format2 := logging.MustStringFormatter(`%{time:15:04:05} %{shortpkg} %{shortfunc} [%{level}] %{message}`)
backend2Formatter := logging.NewBackendFormatter(backend2, format2)
backend2Leveled := logging.AddModuleLevel(backend2Formatter)
// 設定什麼層級的 log 會進到 backend2
backend2Leveled.SetLevel(logging.ERROR, "")

// 註冊
logging.SetBackend(backendFormatter, backend2Leveled)

Mux - API Router

func main() {
    l, _ := net.Listen("tcp", ":3333")
    r := mux.NewRouter()
    r.HandleFunc("/", Index)
    log.Fatal(http.Serve(l, r))
}

func Index(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Time: " + time.Now().Format(time.RFC1123)))
}

Websocket

(Updated on 8th March, 2014)

目前主要 golang 的 websocket 套件有兩個, 分别是官方維護的 code.google.com/p/go.net/websocket,

及非官方版本的 github.com/gorilla/websocket, 我也不知道哪個比較好,

這邊有個比較表可以參考看看

官方版的在使用 go get 下載前還需要先下載 Mercurial 這個版控, 否則無法下載

以下分别使用這兩種套件實現 websocket 的 example

HTML + JS :

<!DOCTYPE html>
<head>
    <title>Test~</title>
</head>
<body>
<script type="text/javascript">
    ws = new WebSocket("ws://54.250.138.78:9090/connws/");
    ws.onopen = function() {
        console.log("[onopen] connect ws uri.");
        var data = {
            "Enabled" : "true"
        };
        ws.send(JSON.stringify(data));
    }
    ws.onmessage = function(e) {
        var res = JSON.parse(e.data);
        console.log(res);
    }
    ws.onclose = function(e) {
        console.log("[onclose] connection closed (" + e.code + ")");
        delete ws;
    }
    ws.onerror = function (e) {
        console.log("[onerror] error!");
    }
</script>
</body>
</html>

[1] 以 go.net/websocket 實作

main :

http.Handle("/connws/", websocket.Handler(ConnWs))
err := http.ListenAndServe(":9090", nil)
if err != nil {
    log.Fatal("ListenAndServe: ", err)
}

func ConnWs(ws *websocket.Conn) {
    var err error
    rec := map[string]interface{}{}

    for {
        err = websocket.JSON.Receive(ws, &rec)
        if err != nil {
            fmt.Println(err.Error())
            ws.Close()
            break
        }
        fmt.Printf("Server received : %v\n", rec)

        if err = websocket.JSON.Send(ws, rec); err != nil {
            fmt.Println("Fail to send message.")
            ws.Close()
            break
        }
    }
}

result :

$ go run main.go
Server received : map[Enabled:true]

[2] 以 gorilla/websocket 實作

main :

http.HandleFunc("/connws/", ConnWs)
err := http.ListenAndServe(":9090", nil)
if err != nil {
    log.Fatal("ListenAndServe: ", err)
}

func ConnWs(w http.ResponseWriter, r *http.Request) :

ws, err := websocket.Upgrade(w, r, nil, 1024, 1024)
if _, ok := err.(websocket.HandshakeError); ok {
    http.Error(w, "Not a websocket handshake", 400)
    return
} else if err != nil {
    log.Println(err)
    return
}

rec := map[string] interface{}{}
for {
    if err = ws.ReadJSON(&rec); err != nil {
        if err.Error() == "EOF" {
            return
        }
        // ErrShortWrite means that a write accepted fewer bytes than requested but failed to return an explicit error.
        if err.Error() == "unexpected EOF" {
            return
        }
        fmt.Println("Read : " + err.Error())
        return
    }
    rec["Test"] = "tt"
    fmt.Println(rec)
    if err = ws.WriteJSON(&rec); err != nil {
        fmt.Println("Write : " + err.Error())
        return
    }
}

result :

$ go run main.go
map[Enabled:true Test:tt]

有特別處理 io error 的 EOF, 否則頁面 refresh 會陷入無窮迴圈

Cross domain

原本是使用官方的版本, 直到有 cross domain 的需求,

HTML5 websocket 本身是支持 cross domain 的,

但是官方版本不知道怎麼去支持它, 一直得到 403,

最後發現 gorilla/websocket 是支持的, 就改用它來達成