On debugging OAuth2 in #golang

Diwaker Gupta -

TL;DR: if you run into strange oauth2 errors especially against not-so-well-known endpoints, you may want to try RegisterBrokenAuthHeaderProvider

While working on some authentication changes for dbxcli, I ran into a strange issue, that ultimately had a simple solution albeit after many frustrated hours of trying to debug the issue. Sharing what I learnt here, with commentary on logging/debugging in Go standard library (or lack thereof)

The Problem

This is the OAuth config I had been using for dbxcli, and it was working just fine:

conf := &oauth2.Config{
		ClientID:     appKey,
		ClientSecret: appSecret,
		Endpoint: oauth2.Endpoint{
			AuthURL:  "https://www.dropbox.com/1/oauth2/authorize",
			TokenURL: "https://api.dropbox.com/1/oauth2/token",
		},
	}

I made a minor change to the config based on the best-practices in the Dropbox API docs:

conf := &oauth2.Config{
		ClientID:     appKey,
		ClientSecret: appSecret,
		Endpoint: oauth2.Endpoint{
			AuthURL:  "https://www.dropbox.com/1/oauth2/authorize",
			TokenURL: "https://api.dropboxapi.com/1/oauth2/token",
		},
	}

Can you notice the difference? That’s it, no other change. Simple, right?

Much to my surprise, the app started erroring out with this cryptic message:

Error: oauth2: cannot fetch token: 400 Bad Request
Response: {“error_description”: “unknown field \”client_id\””, “error”: “invalid_request”}

Huh? What is going on here?

The Journey

  • My first guess was that there was some bug in the server that was triggering due to the change in the API. Doing some manual testing with curl and hand-crafted URLs quickly settled this.
  • My next guess was that I had some other bug in my application code that was not using the OAuth library correctly. To debug this, all I wanted to do was dump all the HTTP requests & responses that happened for the OAuth exchange. Turns out, the Go standard library doesn’t really do any logging and even if it did, there is no way to turn on selective logging. Contrast this with Java, where it is relatively straightforward to log all HTTP requests and responses both from the standard library as well as in popular frameworks like Jetty and Netty. Eventually I had to resort to using tcpdump :(
  • Inspecting the tcpdump did not reveal anything obviously incorrect. So as a last resort, I started going through the oauth2 library source code. And it was then that I came across this gem:
// providerAuthHeaderWorks reports whether the OAuth2 server identified by the tokenURL
// implements the OAuth2 spec correctly
// See https://code.google.com/p/goauth2/issues/detail?id=31 for background.
// In summary:
// - Reddit only accepts client secret in the Authorization header
// - Dropbox accepts either it in URL param or Auth header, but not both.
// - Google only accepts URL param (not spec compliant?), not Auth header
// - Stripe only accepts client secret in Auth header with Bearer method, not Basic
func providerAuthHeaderWorks(tokenURL string) bool {

Aha! Going back to the tcpdump trace, I could see that the request contained both the basic auth header as well as the client secret query parameter.

The Solution

Poking around further, I found this in internal/token.go:

var brokenAuthHeaderProviders = []string{
	"https://accounts.google.com/",
	"https://api.dropbox.com/",
	//... elided for brevity
	"https://www.googleapis.com/",
	"https://www.linkedin.com/",
}

func RegisterBrokenAuthHeaderProvider(tokenURL string) {
	brokenAuthHeaderProviders = append(brokenAuthHeaderProviders, tokenURL)
}

Now we’re getting somewhere! You can see that api.dropbox.com is registered as a brokenAuthHeaderProvider, but api.dropboxapi.com is not! Thankfully, there’s a helpful RegisterBrokenAuthHeaderProvider method!! Putting it all together then, this is what I ended up doing:

func init() {
	// These are not registered in the oauth library by default
	oauth2.RegisterBrokenAuthHeaderProvider("https://api.dropboxapi.com")
	oauth2.RegisterBrokenAuthHeaderProvider("https://api-dbdev.dev.corp.dropbox.com")
}

Problem solved!!

(side note: I find it ironic that almost all the top websites in the world, including Google, are all “broken”. Perhaps the oauth2 lib authors should whitelist the known “good” providers instead?)