Server Setup / Varnish

Solved: Varnish 4 “Internal Server Error 500”

by , , revisited on


We have by far the largest RPM repository with NGINX module packages and VMODs for Varnish. If you want to install NGINX, Varnish, and lots of useful performance/security software with smooth yum upgrades for production use, this is the repository for you.
Active subscription is required.

Why Varnish returns “Internal Server Error 500”

Once you done writing your VCL for Varnish 4, you may not be aware that it silently fails with 500 Internal Server Error for some requests.

Most likely, the reason is client workspace overflow.
When you run varnishlog and watch for requests, you will see: “LostHeader” and “workspace_client overflow”.

You may want to increase workspace_client parameter in varnish.params. Open up /etc/sysconfig/varnish or /etc/varnish/varnish.params and find the line DAEMON_OPTS. Include this: -p workspace_client=128k to solve the issue.

Why is the error actually happening? Most likely it is due your Varnish VCL extensively using regsuball functions to filter out cookies. Instead of increasing workspace_client parameter, a better option is to optimize your VCL to filter all cookies except for the ones you want.

How to confirm that Varnish fails to handle all the cookies? Simple. Test by sending a request with really large cookies set. If your Varnish has problem, you will see 500 Internal Server in the response:

curl -v --cookie "__gads=ID=eb19bcf1ce37f5fe:T=1436110933:S=ALNI_MY2honQXF5C4wn9ppAd7Gi8ITe2kw; __qca=P0-120940762-1436110935090; OX_plg=swf|shk|pm; fofirdId=0884b9ea-3525-4038-80f2-83f83693da0f; jwplayer.volume=1; __distillery=v20150227_7c859bf8-5796-492d-858c-06f2a0b0e5a3; snp_snppopup-exit=1; snp_snppopup-welcome=1; MSFPC=ID=de4505c98afc4bd38013268ab7e2293c&CS=0&LV=201508; SnapABugHistory=1#; hsfirstvisit=http%3A%2F%2Fgetpagespeed.com%2Fwp-admin%2Fadmin.php%3Fpage%3Dshareaholic-settings|http%3A%2F%2Fgetpagespeed.com%2Fwp-admin%2Fupdate-core.php%3Faction%3Ddo-plugin-upgrade|1443153674386; SnapABugVisit=7#1443153673; __hstc=46386168.aab471d96004573413019be5afd42963.1443153674389.1443153674389.1443153674389.1; __hssrc=1; hubspotutk=aab471d96004573413019be5afd42963; hbuid=44498d71-5975-453e-94c6-5397fd6d05fd; hbv_5070=fv%3A1443157102%7Clv%3A1443159156%7Clf%3A0%7Cnv%3A5%7Crf%3A%7Crd%3A%7Cpu%3Agetpagespeed.com%7Cdt%3A2015-09-24%7Cdv%3Acomputer%7Cls%3A0; hbs_5070=190128%7Cnv%3A5%7Cfv%3A1443157103%7Clv%3A1443159156; mlUserID=VDaO7GSQVds9; SS_ARE_Override.traceLevel=WARN; acudeoSession.=%7B%22time%22%3A1445006048750%2C%22adIndex%22%3A5%7D; __smClicked=true; jwplayer.captionLabel=Off; __unam=7639673-14e5ee32f53-5d11080e-982; userid=D44ECB11-9D88-BAE3-B1E4-7F4F3B9C256C; __sonar=5596123696831121725; logglytrackingsession=c21415ab-16b1-4845-8c41-a403a3b97ea9; _inlinkzID=44cfaf4ba2ae; sq_keyword_6099=potluck recipes; sq_keyword_13487=onion burger; sq_keyword_13437=cheesy dip recipes; sq_keyword_13426=pickle stuffed chicken; sq_keyword_13454=rice skillet dinner; sq_keyword_13374=taco tuesday recipes; sq_keyword_6654=easy coquito; sq_keyword_13511=lucky new years recipes; sq_keyword_13517=skillet recipes; sq_keyword_13543=cheese ball recipe; __ar_v4=; sq_keyword_13504=black eyed pea recipe; X-Mapping-fjhppofk=512C68D362FE4F2915D38EE4892A5025; __smDiscoverLayout=list; sq_keyword_13553=easy dinner recipes; __cfduid=d8c77ed62ef7df72f4f916ee5ad0137a81454779527; PHPSESSID=34u0989dqbgmutrbo6h2bgmbk7; sq_keyword_13696=tamale pie recipe; sq_keyword_13981=biscuit recipes; sq_keyword_14004=succotash recipe; sq_keyword_12248=chicken spaghetti; GED_PLAYLIST_ACTIVITY=W3sidSI6Ii8rSDgiLCJ0IjoxNDU1MzgzODE0LCJlZCI6eyJqIjp7IkEiOnsidHQiOjg5MSwicGQiOjg5MSwiYnMiOjEwLCJlcyI6MX19LCJnIjoxNDU1MzgzODEzLCJhIjpbeyJrdiI6e319LHsia3YiOnsiYyI6MSwibSI6NjQyfX0seyJrdiI6e319LHsia3YiOnt9fSx7Imt2Ijp7fX0seyJrdiI6e319XX0sIm52IjoxLCJwbCI6ODkxLCJsdCI6MTQ1NTM4MzgxM30seyJ1IjoiS3pGWSIsInQiOjE0NTUzODI4MjQsImVkIjp7ImoiOnsiQSI6eyJ0dCI6MiwicGQiOjIsImJzIjoxMH19LCJnIjotMSwiYSI6W3sia3YiOnt9fV19LCJudiI6MCwicGwiOjIsImx0IjotMX0seyJ1IjoiOEUrYiIsInQiOjE0NTUzODI3OTYsImVkIjp7ImoiOnsiQSI6eyJ0dCI6OTYsInBkIjo5NiwiYnMiOjEwfX0sImciOi0xLCJhIjpbeyJrdiI6e319LHsia3YiOnsibSI6MX19LHsia3YiOnt9fSx7Imt2Ijp7fX0seyJrdiI6e319LHsia3YiOnt9fV19LCJudiI6MCwicGwiOjk2LCJsdCI6LTF9XQ..; sq_keyword_14016=stuffed baked potatoes; gsScrollPos=; sq_keyword_13966=vegetarian recipes; sq_keyword_10299=banana bread recipes; sq_type=img; __atssc=google%3B12%2Creddit%3B2%2Cpinterest%3B1; __zlcmid=ZDfKlmSP5E9GBM; __uis=97356709-90f4-596a-b51b-ca1aaa2ea460; wordpress_test_cookie=WP+Cookie+check; duo_wordpress_auth_cookie=ZHVvX3dvcmRwcmVzc19hdXRoX2Nvb2tpZXxkMlZoY25samFHVm18UkVsWk4wTlZORWRZUmpOVVEwMU1OMHRZT0VNPXwxNDU2MDI5NDQy%7C0b2d591a0a510b63cbdce4b31528ecf5a54cb560; wordpress_logged_in_1367953aea58bdf694223267a977a9dc=getpagespeed%7C1456029442%7CU8DfymeFu6N8oRzCxgP501cTLuWwKXPiAXIjhDEphWX%7C0f84b8e22375bee5deb514176341fd894d73353449143ab04bc8898134681dd2; crtg_rta=300250m%3B300250ds%3Bf300600atf%3Bf300600btf%3Bf72890%3Bf970250%3Bf300250%3Bf160600%3Bf300250r%3Bf970250t2%3Bf300600bt2%3B300600t3%3B160600t3%3Bf300600bt3%3Bf300250rt2%3Bf300250rt3%3B300250t2%3Bf300250mr%3Bf300250mc%3Bm300250btf%3Bm300600t2%3Bm300600t3%3Bm970250%3Bm970250t2%3Bm300250m%3Bm32050m%3B72890t1%3B72890t2%3Bf72890t1%3Bf72890t2%3B; amplitude_idgetpagespeed.com=eyJkZXZpY2VJZCI6IjYzZjE3NDI4LTBiNDEtNGNkMi04YzZlLTkwNDkwMTBkN2Q0ZCIsInVzZXJJZCI6bnVsbCwib3B0T3V0IjpmYWxzZX0=; __ybotu=ikvhgqxvy248h8h1wh; __ybotv=1455994386786; __atuvc=1%7C7; wp-settings-1=editor%3Dhtml%26hidetb%3D1%26unfold%3D1%26align%3Dcenter%26imgsize%3Dfull%26libraryContent%3Dbrowse%26wplink%3D1%26urlbutton%3Dnone%26advImgDetails%3Dshow%26dfw_width%3D822%26posts_list_mode%3Dlist%26post_dfw%3Doff; wp-settings-time-1=1455994752; _gat_tw=1; _ga=GA1.2.115856649.1455994385" https://www.getpagespeed.com/abcde
HTTP/1.1 500 Internal Server Error
  1. Shawn Corrigan

    I’m annoyed because it was obvious SEO clickbait with a “Solved: ” prefix, but what I’m even more annoyed about is that it actually fixed my issue.

    Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: