Friends, the update to 1.7 broke the work with IP addresses of clients.
I am proxying Go through Nginx
location /backend/ {
rewrite /backend/(.*) /$1 break;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://127.0.0.1:8080;
}
I read this https://github.com/gin-gonic/gin/issues/2693
But I don't understand what to write in router.TrustedProxies
func GinRouter() *gin.Engine {
router := gin.New()
// Set a lower memory limit for multipart forms
router.MaxMultipartMemory = 100 << 20 // 100 MB
// Custom Logger
router.Use(gin.LoggerWithFormatter(func(param gin.LogFormatterParams) string {
return fmt.Sprintf("%s |%s %d %s| %s |%s %s %s %s | %s | Body: %s | %s | %s\n",
param.TimeStamp.Format(time.RFC1123),
param.StatusCodeColor(),
param.StatusCode,
param.ResetColor(),
param.ClientIP,
param.MethodColor(),
param.Method,
param.ResetColor(),
param.Path,
param.Latency,
byteCountSI(param.BodySize),
param.Request.UserAgent(),
param.ErrorMessage,
)
}))
// Recovery middleware recovers from any panics and writes a 500 if there was one.
router.Use(gin.Recovery())
return router
}
Thanks everyone 🙏🏻
Comment From: cyanBone
me too
Comment From: 1989bajie
框架升级是不是需要考虑平滑？这就意味着用过这个方法的用户，升完级都要出问题了。。。。
Comment From: leungyauming
框架升级是不是需要考虑平滑？这就意味着用过这个方法的用户，升完级都要出问题了。。。。
Translate: Shouldn't migration be considered in upgrades? That means users who have used the method will be in trouble after upgrading.
Comment From: artlevitan
The best way is roll back the trusted proxy update and make it optional.
Almost everyone uses Nginx and add-on packages so that there is no doubt about the client.
Comment From: leungyauming
// ClientIP implements a best effort algorithm to return the real client IP.
// It called c.RemoteIP() under the hood, to check if the remote IP is a trusted proxy or not.
// If it's it will then try to parse the headers defined in Engine.RemoteIPHeaders (defaulting to [X-Forwarded-For, X-Real-Ip]).
// If the headers are nots syntactically valid OR the remote IP does not correspong to a trusted proxy,
// the remote IP (coming form Request.RemoteAddr) is returned.
According to the documentation of the gin.Context.ClientIP() function, it will check if the remote IP (which is the ip of the proxy or the ip of nginx in your case) is a trusted IP or not.
If it is a trusted IP (which means the request is redirected by a proxy), then it will try to parse the "real user IP" from X-Forwarded-For/X-Real-Ip header.
Conclusion: You can think TrustedProxies as the identifiers for classifying whether the request is redirected from a proxy or not. If you don't type any IP into TrustedProxies, then you may get the IP of the proxy as the "user" IP.
For Chinese:
根據gin.Context.ClientIP()函數的文檔，該函數會先透過TrustedProxies變量來偵查HTTP請求的來源IP是否是所謂的“信任代理”。
如果該HTTP請求的來源IP屬於“信任代理”那麼伺服器就會嘗試從X-Forwarded-For或X-Real-Ip頭文取得真正用戶的IP。所以簡單來說就是，TrustedProxies是用來給伺服器判斷是否從頭文取得IP的。
Comment From: artlevitan
Thanks for the answer, I saw the documentation and code comments.
Why is 127.0.0.1 not the default trusted address? Most people use Nginx for serve to WAN.
Comment From: leungyauming
// New returns a new blank Engine instance without any middleware attached.
// By default the configuration is:
// - RedirectTrailingSlash: true
// - RedirectFixedPath: false
// - HandleMethodNotAllowed: false
// - ForwardedByClientIP: true
// - UseRawPath: false
// - UnescapePathValues: true
func New() *Engine {
debugPrintWARNINGNew()
engine := &Engine{
RouterGroup: RouterGroup{
Handlers: nil,
basePath: "/",
root: true,
},
FuncMap: template.FuncMap{},
RedirectTrailingSlash: true,
RedirectFixedPath: false,
HandleMethodNotAllowed: false,
ForwardedByClientIP: true,
RemoteIPHeaders: []string{"X-Forwarded-For", "X-Real-IP"},
TrustedProxies: []string{"0.0.0.0/0"},
AppEngine: defaultAppEngine,
UseRawPath: false,
RemoveExtraSlash: false,
UnescapePathValues: true,
MaxMultipartMemory: defaultMultipartMemory,
trees: make(methodTrees, 0, 9),
delims: render.Delims{Left: "{{", Right: "}}"},
secureJSONPrefix: "while(1);",
}
engine.RouterGroup.engine = engine
engine.pool.New = func() interface{} {
return engine.allocateContext()
}
return engine
}
As you can see, the default value of the TrustedProxies field is []string{"0.0.0.0/0"} which should has included 127.0.0.1 already.
And alternative IP in X-Forwarded-For/X-Real-IP should be returned if they actually present.
Comment From: artlevitan
it does not work for me, as well as for many who opened topics with such a problem.
https://github.com/gin-gonic/gin/issues/2693
Comment From: leungyauming
Interesting, let me go to take a look and test it.
Comment From: artlevitan
log from SSH
main[25495]: Thu, 29 Apr 2021 12:16:51 | 200 | 127.0.0.1 | ....
main[25495]: Thu, 29 Apr 2021 12:16:53 | 200 | 127.0.0.1 | ....
In all responses localhost (127.0.0.1)
the proxy configuration was thrown above
location /backend/ {
rewrite /backend/(.*) /$1 break;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://127.0.0.1:8080;
}
Comment From: leungyauming
client/main.go
package main
import (
"fmt"
"io"
"net/http"
)
func main() {
req, _ := http.NewRequest(http.MethodGet, "http://127.0.0.1:8080", nil)
req.Header.Add("X-Forwarded-For", "123.123.123.123")
res, _ := http.DefaultClient.Do(req)
body := res.Body
data, _ := io.ReadAll(body)
fmt.Println(string(data))
defer body.Close()
}
server/main.go
package main
import (
"net/http"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.New()
r.GET("/", func(c *gin.Context) {
c.String(http.StatusOK, "%s", c.ClientIP())
})
r.Run()
}
I created a simple client and server.
Then sent a http request with X-Forwarded-For header set to 123.123.123.123 from the client to the server.
Print the result to the console.
Here is the output. As you can see, the result is correct and the feature is working.
May you please check if the header is set and sent to the server successfully? (I suggest printing the headers received in your handler function, the variable name is c.Request.Header which should be a string-string map)
Comment From: artlevitan
Header X-Forwarded-For from Nginx in my case is correct too But c.ClientIP() in GIN 1.7 always return me 127.0.0.1
If I use
req.Header.Get("X-Forwarded-For")
I get the correct address too, like your example
I can't understand how to work with Nginx & GIN 1.7+
Comment From: artlevitan
Just in case, I post the source code to check
package main
import (
"fmt"
"net/http"
"os"
"time"
"github.com/gin-gonic/gin"
)
// Response
type Response struct {
Status int `json:"status"`
Message string `json:"message,omitempty"`
}
// ResponseBody
type ResponseBody struct {
Response `json:"response"`
Additional interface{} `json:"additional,omitempty"`
Payload interface{} `json:"payload,omitempty"`
}
// GinRouter - GIN-Router
func GinRouter() *gin.Engine {
router := gin.New()
// Set a lower memory limit for multipart forms
router.MaxMultipartMemory = 100 << 20 // 100 MB
// Custom Logger
router.Use(gin.LoggerWithFormatter(func(param gin.LogFormatterParams) string {
return fmt.Sprintf("%s |%s %d %s| %s |%s %s %s %s | %s | %s | %s\n",
param.TimeStamp.Format(time.RFC1123),
param.StatusCodeColor(),
param.StatusCode,
param.ResetColor(),
param.ClientIP,
param.MethodColor(),
param.Method,
param.ResetColor(),
param.Path,
param.Latency,
param.Request.UserAgent(),
param.ErrorMessage,
)
}))
// Recovery middleware recovers from any panics and writes a 500 if there was one.
router.Use(gin.Recovery())
return router
}
func main() {
// Server Settings
router := GinRouter()
server := &http.Server{
Addr: ":8080",
Handler: router,
ReadTimeout: 30 * time.Second,
WriteTimeout: 30 * time.Second,
MaxHeaderBytes: 1 << 20,
}
// Ping test
router.GET("/ping", func(c *gin.Context) {
var respBody ResponseBody
c.JSON(respBody.Status, respBody)
})
//
err := server.ListenAndServe()
if err != nil {
fmt.Println("Could not start application")
fmt.Println(err.Error())
os.Exit(1)
}
}
Comment From: leungyauming
Did you deploy your Nginx to the same machine as the server and the client?
Header X-Forwarded-For from Nginx in my case is correct too But c.ClientIP() in GIN 1.7 always return me 127.0.0.1
If I use
req.Header.Get("X-Forwarded-For")I get the correct address too, like your exampleI can't understand how to work with Nginx & GIN 1.7+
I know you said that you can get the correct address through header.
But just want to make sure, because the $remote_addr variable will be 127.0.0.1 if you send a request to the Nginx proxy with the same machine.
Comment From: artlevitan
GO and Nginx are on the same server, everything was ok before the update gin 1.7
Comment From: leungyauming
Try to send a request to Nginx with another machine, because if you send the request using the same machine with Nginx, Nginx will set the $remote_addr variable of the proxy_set_header procedure to be 127.0.0.1 and then your Go application will receive 127.0.0.1 as the final IP of the "user".
Comment From: artlevitan
Are you proposing to change the server infrastructure, which works fine due to the fact that the GIN version has been updated?
Comment From: leungyauming
No, I didn't mean you need to change the server infrastructure. I just want to make sure that you are actually testing it in the right way.
- You said your Go application and your Nginx proxy server is deployed in the same machine
- You said getting the IP through the header is working but through the Nginx proxy is not working
- I'm not sure if you are using the same machine to send HTTP request to your Nginx proxy (because if you send a request to Nginx with the same machine your IP will be
127.0.0.1)
As the result, I think it may be caused by the 3 point I just mentioned. So I call you to try to "send request" to your Nginx proxy with a different machine, I didn't mean moving the Golang application to another machine.
Comment From: xpunch
I meet the same problem too, after debug the application, I found this issue occur when engine.Run() was not called. My application use gin as web handler, so it didn't need to startup by engine.Run, which makes trustedCIDRs not got initialized. So it will never read IP from headers, case trustedCIDRs is nil.
Comment From: leungyauming
I meet the same problem too, after debug the application, I found this issue occur when engine.Run() was not called. My application use gin as web handler, so it didn't need to startup by engine.Run, which makes trustedCIDRs not got initialized. So it will never read IP from headers, case trustedCIDRs is nil.
Good catch! I didn't actually notice that too.
Comment From: yiranzai
well, I pointed it out at the beginning. https://github.com/gin-gonic/gin/pull/2692
Comment From: leungyauming
@artlevitan Please take a look and sorry for my misleading information.
Comment From: evanfuller
I have also run into this problem. I see there is a PR to expose a workaround for setting trusted proxies, but I can't tell if it was abandoned.
Comment From: wackwinds
when we start with RunTLS instand of Run, ClientIP() will always return 127.0.0.1 while prepareTrustedCIDRs() not called it seems ClientIP just work fine under http, not https or others
Comment From: wackwinds
看了下源码，prepareTrustedCIDRs只在run里有调用，RunTLS没调，也就是ClientIP只能在http环境使用，上面某评论说一直返回127.0.0.1其实也是因为走的https环境
Comment From: SteadyWorkerS
nginx and gin v1.7 same machine, and send request to nginx from another machine, but method ClientIP return "127.0.0.1", which is not we want
Comment From: SennoYuki
这个升级的破坏性真是神坑
Comment From: axiaoxin
@artlevitan try to use app.SetTrustedProxies function. I can get the real client IP, but still will get ::1 sometimes.
@yiranzai Gin server behind nginx still can't get real client IP correctly with the newest master branch code.
Nginx conf added headers:
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
gin server add code with the newest master gin version
app.ForwardedByClientIP = true
app.SetTrustedProxies([]strings{"127.0.0.0/8", "192.168.0.0/16", "172.16.0.0/12", "10.0.0.0/8"})
app.RemoteIPHeaders = []string{"X-Forwarded-For", "X-Real-IP"}
Sometimes get the real IP correctly, but will sometimes get ::1.
Comment From: yiranzai
@axiaoxin ::1 ipv6?
Comment From: axiaoxin
@axiaoxin
::1ipv6?
Should be, but nginx log the $remote_addr is a correct ipv4
Comment From: larryclean
Why only the Run method executes trustedCIDRs, err := engine.prepareTrustedCIDRs().Open external access for engine.prepareTrustedCIDRs()?
Comment From: bilinxing
That prepareTrustedCIDRs() only in the Engine.Run() call , but the Engine.Run() method is not really available in the production environment. Bad and very destructive upgrade！！！
Comment From: petanne
Bad and very destructive upgrade！！！
Comment From: montanaflynn
@axiaoxin I had same problem, I used router.TrustedProxies = []string{"0:0:0:0:0:0:0:0001"} and it worked though.
@leungyauming When will the fix for code that doesn't use router.Run be released? This is basically all production code that uses graceful termination and is behind trusted proxies like cloudfront.
Comment From: montanaflynn
By the way I ended up just using this function since I wanted the old logic and my servers are behind cloudfront:
func getClientIP(c *gin.Context) string {
forwardHeader := c.Request.Header.Get("x-forwarded-for")
firstAddress := strings.Split(forwardHeader, ",")[0]
if net.ParseIP(strings.TrimSpace(firstAddress)) != nil {
return firstAddress
}
return c.ClientIP()
}
Then replaced c.ClientIP() with getClientIP(c) and I'm back in business. Really unfortunate that just upgrading version caused us to lose days of user's real IP addresses, this should have been an opt-in change in my opinion, especially since it was broken.
Comment From: kainosnoema
Also having this issue. We don't call engine.Run() and instead just use the handler directly. When will this get addressed?
Comment From: AlbinoGeek
Also waiting for this to be fixed...
Without that method exposed, I can't enable the feature, as I don't start the server using .Run()
Comment From: dpanic
By the way I ended up just using this function since I wanted the old logic and my servers are behind cloudfront:
go func getClientIP(c *gin.Context) string { forwardHeader := c.Request.Header.Get("x-forwarded-for") firstAddress := strings.Split(forwardHeader, ",")[0] if net.ParseIP(strings.TrimSpace(firstAddress)) != nil { return firstAddress } return c.ClientIP() }Then replaced
c.ClientIP()withgetClientIP(c)and I'm back in business. Really unfortunate that just upgrading version caused us to lose days of user's real IP addresses, this should have been an opt-in change in my opinion, especially since it was broken.
I used your code, but modified it to be as a middleware which I invoke before all.
I did it this way in order to avoid wrapping and changing everywhere ctx.ClientIP()
func Preprocess(logger *zap.Logger) gin.HandlerFunc {
return func(ctx *gin.Context) {
_, port, _ := net.SplitHostPort(strings.TrimSpace(ctx.Request.RemoteAddr))
ip := getClientIP(ctx)
ctx.Request.RemoteAddr = fmt.Sprintf("%s:%s", ip, port)
ctx.Next()
}
}
Of course this is only mitigation, we're all waiting for regular fix of this issue.
Comment From: guillermo-menjivar
any updates on this issue?
Comment From: adamwoolhether
just discovered issues wtih this today as well....
Comment From: virskor
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HTTP_X_FORWARDED_FOR $remote_addr;#在多级代理的情况下，记录每次代理之前的客户端真实ip
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
proxy_pass https://remote_app_proxy_url;
after add NGINX configurations use code below to get real client IP
also Gin egine
router := gin.New()
router.TrustedProxies := []string{"0.0.0.0/0"}
router.RemoteIPHeaders := []string{"X-Forwarded-For", "X-Real-IP"}
func GetRealIP(c *gin.Context) string {
ip := c.GetHeader("X-Real-IP")
if len(ip) == 0 {
return c.ClientIP()
}
return ip
}
will this comment help you? i hope that
Comment From: artlevitan
Hey! No, it didn't help me, I tried this method at the beginning.
Comment From: artlevitan
I use this code until Gin is updated normally.
func clientIP(c *gin.Context) string {
ip := c.GetHeader("X-Forwarded-For") // ip := c.ClientIP()
// localhost
if ip == "127.0.0.1" || ip == "::1" {
return ""
}
// Drop Local / Reserved Address Ranges
var (
ip1_1 = net.ParseIP("10.0.0.0")
ip1_2 = net.ParseIP("10.255.255.255")
ip2_1 = net.ParseIP("127.0.0.0")
ip2_2 = net.ParseIP("127.255.255.255")
ip3_1 = net.ParseIP("169.254.0.0")
ip3_2 = net.ParseIP("169.254.255.255")
ip4_1 = net.ParseIP("172.16.0.0")
ip4_2 = net.ParseIP("172.31.255.255")
ip5_1 = net.ParseIP("192.168.0.0")
ip5_2 = net.ParseIP("192.168.255.255")
)
trial := net.ParseIP(ip)
if bytes.Compare(trial, ip1_1) >= 0 && bytes.Compare(trial, ip1_2) <= 0 {
return ""
}
if bytes.Compare(trial, ip2_1) >= 0 && bytes.Compare(trial, ip2_2) <= 0 {
return ""
}
if bytes.Compare(trial, ip3_1) >= 0 && bytes.Compare(trial, ip3_2) <= 0 {
return ""
}
if bytes.Compare(trial, ip4_1) >= 0 && bytes.Compare(trial, ip4_2) <= 0 {
return ""
}
if bytes.Compare(trial, ip5_1) >= 0 && bytes.Compare(trial, ip5_2) <= 0 {
return ""
}
return ip
}
Comment From: thinkerou
v1.7.7 have released, thanks! https://github.com/gin-gonic/gin/releases
Comment From: bytejedi
Who cares about Trusted Proxies and Trusted shit... The bugs introduced by this feature are ridiculous. That's an env security stuff. This is the job of a load balancer or something like that in front of the gin server. Not suitable for enterprise users
Comment From: jiyeyuran
Still not working if remote IP is ::1 .
Comment From: jiyeyuran
engine := gin.New()
engine.SetTrustedProxies([]string{"0.0.0.0/0", "::/0"})
Comment From: duaneking
I just bumped into this; this "feature" is full of defects, imho.
Comment From: ECHibiki
just set trusted proxies []string to have ::1 and get nginx to pass down the real ip...