Today at work, a feature I worked on went from fun and cool as I was building it to a stressful fix and recover operation after it unintentionally deleted important site files when deployed.
Continue reading post "#2820"cache posts
I was getting a white screen with 200 status in production after updating Symfony to 3.4.
Continue reading post "#1696"Logging service worker cache headers
As part of the service worker API, a cache interface has been provided to manage cached request-response pairs. In working on the service worker for my site, I wanted to see what headers the cached requests and responses had, but due to the asynchronous way many of the cache properties are accessed, this was a bit verbose. I wrote out a script that I could paste in the JS console to look at all stored request-response pairs in a given cache so I could examine them:
caches.open('cache-name').then(function(_cache){
_cache.keys().then(function(_keys){
_keys.forEach(function(_request){
var _requestLog = [];
_requestLog.push(['request', _request.url, _request]);
_request.headers.forEach(function(){
_requestLog.push(['request header', arguments]);
});
_cache.match(_request).then(function(_response){
_requestLog.push(['reponse', _response]);
_response.headers.forEach(function(){
_requestLog.push(['response header', arguments]);
});
}).then(function(){
_requestLog.forEach(function(_item){
console.log.apply(console, _item);
});
});
});
});
});
Replace cache-name with whatever key you’re using for your cache. Be warned that this will produce a long log if you’ve got more than a few items in the cache. You can also see just the requests you have a cache for with something like:
caches.open('cache-name').then(function(_cache){
_cache.keys().then(function(_keys){
_keys.forEach(function(_request){
console.log(['request', _request.url, _request]);
});
});
});
First play with service workers
I started playing with service workers as a client side cache manager a bit tonight. I’m using this Smashing Magazine article as a guide. I’ve been reading a bit here and there about them, intrigued by their role in making web sites installable as apps and their ability to allow sites to function even while offline. However, my site’s current lack of pages and other priorities plus the learning curve and things that have to be done to set them up kept me from playing with them until now.
Workers require HTTPS, unless, luckily, you are serving from localhost. I had to modify my local app install to use that instead of the more site-indicative name it was using. They also require placement at or above the path level they apply to, or theoretically a Service-Worker-Allowed header, though I was unable to get that working. I’m assuming this is for some security reason. Because my file is stored in a Symfony bundle and because I am serving multiple sites with the same application, I didn’t want an actual file in my web root. I made a Symfony route and action that passes through the file, like:
Ideas: Local + Proxy Remote Hosting for Personal Site
Hosting your personal website on a computer at your home puts extra indie in indieweb. You truly control all of your data. I did this for several years. I did this with a very modest setup, serving from a mobile home using an iBook G3 800 with Windstream DSL internet. Performance obviously wasn’t the same as a web host would’ve provided. Of course, it helped a lot that I didn’t have much traffic. But I still had a lot of downtime, for a number of reasons:
- Dynamic IPs: most consumer level internet service plans do not have a static IP, and change occasionally. I used DynDNS to accomodate this, but it still led to downtime between the time that the IP changed to the time the daemon was run, DynDNS updated its records, and the DNS propagated.
- Internet outages: consumer level plans definitely don’t have the robust connection that a web host has. This was especially true at my mobile home, where perhaps old wiring led to fairly frequent outages, especially on windy days.
- Power outages: hosting companies have backup power. Most homes do not. My power went out from the electric company at least several times while I was hosting, but also went out whenever I had to turn off the power to work on something electrical. My server would stay on because it was a laptop, but not the router. A UPS is a reasonably priced option for reducing or eliminating this problem though.
- Computer / router issues, updates, etc: Any reboots, shutdowns, or stopping of server daemons will mean your site is down, which could be needed for updates or various problems. Web hosts usually have robust servers, and if they’re managing them, they’re usually very good about keeping them up and doing updates quickly and during down-times.
My idea to mitigate performance and downtime problems would be to use a reverse proxy, such as varnish, running on a remote web host, with your DNS pointing at it. It would be configured to go to your home server’s IP for content. You’d have to set up a daemon to contact the remote server and update this when it changes. Public pages would be set with long cache times so that they would be available if your home server goes down. The application(s) on the server would then have to be set up to send a PURGE request when pages were updated. Or perhaps, if the proxy allows, you could use whatever maxage times you want but have the proxy store the cached responses indefinitely and server them if the home server can’t be reached even if the maxage has been passed.
This idea is not without its problems. For instance:
- Security of connection between servers: If your site is using SSL, the connection between the servers would also have to be over SSL or the SSL used between the client and remote server would be virtually worthless. Without SSL between the two, a man in the middle could easily eavesdrop on the traffic or divert all traffic to their own server. Because of the changing IP address, the home server would have to use a self-signed certificate possibly increasing the risk of a man in the middle attack between the two servers and at the least requiring the remote server to accept that cert from any IP that it considers your home server.
- Non-cacheable requests would always need the home server: Private pages like admin pages as well as any mutating (POST, etc.) requests, would always have the same performance and robustness issues as the home server. Most importantly for many personal sites, webmentions / pingbacks / trackbacks / comment submissions would fail if the home server went down. So would any other form submissions. To deal with this, you’d probably have to do some programming on the remote server to have it queue these requests and give it an appropriate generic response for the request. For admin and logged in user activity, you could build the client side of your app to operate as you desire in offline mode.
And, as is always the case with serving from home, server and home network configuration, security, maintenance, etc. is all on you. There isn’t really a “managed” option available. You’ll have to get everything working, apply updates, deal with server and network problems, etc. In a home environment, security also includes physical access to the device.
Symfony AppCache and ‘X-Reverse-Proxy-TTL’, a hack
Symfony’s HttpCache reverse proxy is a simple way to get caching of pages with Symfony. It is simple to set up, easy to work with, and easy to clear. I started using it recently on my own site.
A simple app/console cache:clear will clear the entire cache. Otherwise, following the HTTP-oriented spirit of the framework, invalidation is based entirely on HTTP headers. In this way, it works the same as proxy / gateway caches. It only caches responses with public Cache-Control headers. It is age based, using the Cache-Control s-maxage or maxage values or Expires headers (following that order of precedence). It then considers the cached items fresh until they are stored for longer than those headers specify they can be stored. The cached version is served, bypassing the router / controller, as long as the cache is fresh.
This is all nice, but using long max-ages for those headers means that caches outside of my control can cache pages for long periods of time. cache:clear won’t help when a page changes. One possible option would be to have shorter and safer max-ages as Cache-Control headers and use something else for HTTPCache.
Symfony AppCache: built in reverse HTTP proxy
I finally set up my site to work with Symfony’s built in HTTP reverse proxy. Took a little bit of time since I had to fix a couple minor bugs in how things are set up with my symfony-initial and Symfony Standard Edition Bundle and then made a mistake in testing whether or not it was working that made me think it wasn’t when it was.
One useful way to test if it’s working is to set the ‘debug’ option of AppCache to true (turn this back off for production). This will set an X-Symfony-Cache header that will provide info on the cache behaviour. You can see these headers on the shell by running curl -I your.url. If it says ‘fresh’ as part of the header value, that means it was served from the cache. If it shows the header at all, that means AppCache is being used.
For the cache to work, the response must be public and have something set to control how the cache becomes stale. See Symfony’s docs on caching for more details. Since my content rarely changes at the moment, I went with the Cache-Control header with max-age. A cool thing about using Symfony’s reverse proxy is that the entire cache will be cleared when clearing Symfony’s cache like normal. This means that if you make a mistake and must remove it from the cache, there is a quick and easy way.