Friday, November 13, 2009

Curl has a bug with redirects?

I gave a presentation at the JUG last night about JAX-RS and Jersey. The talk went well, and it was a great crowd with a lot of participation (I'll post more once I have the materials cleaned up and they are posted to the JUG website).

But one thing that didn't go right last night was when I was using the 'curl' command line tool to create new artists in the RESTful music database service I was using as an example.

The curl man page on my MacBook Pro says:

When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will do the following request with a GET if the HTTP response was 301, 302, or 303. If the response code was any other 3xx code, curl will re-send the following request using the same unmodified method.

Yet, when I was using it, it wasn't doing that. For example when trying to create artist 'Jane':


brian@Widget: curl -v -H "Accept: text/html" -X POST -L "http://localhost:3131/artists/Jane"
* About to connect() to localhost port 3131 (#0)
* Trying ::1... Connection refused
* Trying fe80::1... Connection refused
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 3131 (#0)
> POST /artists/Jane HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: localhost:3131
> Accept: text/html
>
< HTTP/1.1 303 See Other
< server: grizzly/1.9.10
< Location: http://localhost:3131/artists/id/313
< Content-Type: text/plain; charset=iso-8859-1
< Content-Length: 0
< Date: Fri, 13 Nov 2009 05:24:12 GMT
<
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:3131/artists/id/313'
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 3131 (#0)
> POST /artists/id/313 HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: localhost:3131
> Accept: text/html
>
< HTTP/1.1 405 Method Not Allowed
< server: grizzly/1.9.10
< Allow: DELETE,OPTIONS,HEAD,GET
< Content-Type: text/plain; charset=iso-8859-1
< Content-Length: 0
< Date: Fri, 13 Nov 2009 05:24:12 GMT
<

When it followed the redirect and issued the follow-up request, it went to the right place:


...
< HTTP/1.1 303 See Other
< server: grizzly/1.9.10
< Location: http://localhost:3131/artists/id/313
...
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:3131/artists/id/313'

But it didn't switch to GET:

> POST /artists/id/313 HTTP/1.1

As a result, I got a 405 error code because curl tried to do a POST to the redirected URL rather than doing a GET as is documented.

I did some further experiments last evening after the talk and confirmed this behavior. I even tried switching the return code to a 302 just to see if it would make a difference and it doesn't.

How does your curl behave? Does it switch from POST to GET when following a redirect using the -L option?

I'd appreciate it if anyone who knows anything about this could comment. Is curl doing the wrong thing, or is the man page wrong and curl is no longer supposed to do this?

4 comments:

Anonymous said...

For other users discovering this exact same and very annoying behavior.

As per the bug report below this is currently seen as a feature. And the -X is only a way of hard code the method in all curl operations.

To achieve the correct behavior when posting to a form that results in a redirect you must use the -F or -d flags instead.

See this old curl bug report:
http://sourceforge.net/tracker/index.php?func=detail&aid=1807732&group_id=976&atid=100976

Brian Gilstrap said...

Excellent! Thanks so much for providing that clarification, 'Anonymous'. Re-reading the curl man page the value of -d and -F options for these purposes becomes much clearer.

I do wish they had made it clearer -X forces the method to be POST even with redirects.

Anonymous said...

Thanks a lot !
I've just encounter the exact same problem.

As curl provide the "--post303" options for forcing this behavior, I could not imagine it was a bug ...

Unknown said...

I think it is a bug, since curl does not provide a special command for DELETE Method. So it is not able to follow a 303 after DELETE with a GET request...