{"id":1556,"date":"2015-07-05T22:33:42","date_gmt":"2015-07-05T13:33:42","guid":{"rendered":"http:\/\/kazu.tv\/blog\/?p=1556"},"modified":"2015-07-05T23:57:32","modified_gmt":"2015-07-05T14:57:32","slug":"postgresql-backup-tool-wal-e-is-useful","status":"publish","type":"post","link":"https:\/\/kazu.tv\/blog\/2015\/07\/05\/postgresql-backup-tool-wal-e-is-useful\/","title":{"rendered":"PostgreSQL \u306e\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u30c4\u30fc\u30eb\uff1f wal-e \u304c\u4fbf\u5229"},"content":{"rendered":"<h2>PostgreSQL \u306e\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u3092\u3057\u305f\u3044<\/h2>\n<p>\u4eca\u81ea\u5206\u305f\u3061\u3067\u4f5c\u3063\u3066\u3044\u308b<a href=\"http:\/\/issf.hidebu.info\/\" target=\"_blank\">\u30b5\u30fc\u30d3\u30b9<\/a>\u3067\u306f\u3001\u6210\u308a\u884c\u304d\u4e0a PostgreSQL \u3092\u4f7f\u3063\u3066\u3044\u308b\u3002\u3067\u3001\u6700\u8fd1\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306e\u8a2d\u5b9a\u3092\u3057\u305f\u3093\u3060\u3051\u3069\u3001\u4eca\u307e\u3067\u3068\u540c\u3058\u304f pg_dump \u3092\u3069\u3063\u304b\u5225\u306e\u5834\u6240\u306b\u4fdd\u5b58\u3059\u308b\u3063\u3066\u306e\u3060\u3068\u3064\u307e\u3089\u306a\u3044\u306a\u3068\u601d\u3063\u3066\u3061\u3087\u3063\u3068\u3050\u3050\u3063\u3066\u307f\u305f\u3089 <a href=\"https:\/\/github.com\/wal-e\/wal-e\" target=\"_blank\">wal-e<\/a> \u3063\u3066\u306e\u304c\u3042\u3063\u305f\u3002\u7d50\u8ad6\u304b\u3089\u66f8\u304f\u3068\u3001\u3053\u308c\u304b\u306a\u308a\u4fbf\u5229\u3002<\/p>\n<p>\u305f\u3060\u3057\u3001\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u306f\u3042\u307e\u308a\u5206\u304b\u308a\u3084\u3059\u3044\u3068\u306f\u8a00\u3048\u306a\u3044\u3002<\/p>\n<h2>wal-e \u304c\u3084\u3063\u3066\u304f\u308c\u308b\u3053\u3068<\/h2>\n<h3>\u5927\u96d1\u628a\u306b\u306f<\/h3>\n<p>\u8981\u306f PITR (Point In Time Recovery) \u306e\u305f\u3081\u306e\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u304c\u53d6\u5f97\u3067\u304d\u308b\u3002PostgreSQL \u306e PITR \u306b\u95a2\u3057\u3066\u306f\u3001\u516c\u5f0f\u30b5\u30a4\u30c8\u306e<a href=\"http:\/\/www.postgresql.org\/docs\/9.4\/static\/continuous-archiving.html\" target=\"_blank\">\u30da\u30fc\u30b8<\/a>\u3092\u53c2\u7167\u3002<\/p>\n<p>\u5177\u4f53\u7684\u306b\u306f\u3001<\/p>\n<ul>\n<li>\u30d9\u30fc\u30b9\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306e\u53d6\u5f97<\/li>\n<li>WAL (Write Ahead Log) \u306e\u53d6\u5f97<\/li>\n<\/ul>\n<p>\u3092\u3057\u3066\u304f\u308c\u308b\u3002\u3067\u3001\u305d\u308c\u3092 Amazon S3, Windows Azure Blob Service, OpenStack Swift \u306b\u4fdd\u5b58\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u3002<\/p>\n<p>\u30ea\u30b9\u30c8\u30a2\u3082\u3001\u6700\u65b0\u3078\u306e\u30ea\u30b9\u30c8\u30a2\u3001\u3042\u308b\u3044\u306f\u904e\u53bb\u3078\u306e\u30ea\u30b9\u30c8\u30a2\u304c\u51fa\u6765\u308b\u3002<\/p>\n<h3>\u5c11\u3057\u7d30\u304b\u3044\u4ed5\u7d44\u307f\u306a\u3069<\/h3>\n<p>\u30d9\u30fc\u30b9\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306f\u3001\u30e6\u30fc\u30b6\u30fc\u304c\u30b3\u30de\u30f3\u30c9\u3067\u8d77\u52d5\u3059\u308b\u3082\u306e\u3067\u3001\u901a\u5e38\u306f crontab \u306a\u3069\u306b\u8a2d\u5b9a\u3059\u308b\u3002\u30d9\u30fc\u30b9\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306e\u53d6\u5f97\u524d\u5f8c\u3067\u3001begin backup, end backup \u3092\u5b9f\u884c\u3057\u3066\u304f\u308c\u3001\u307e\u305f\u3001\u30d9\u30fc\u30b9\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306e\u5185\u5bb9\u306fS3\u306a\u3069\u306b\u81ea\u52d5\u7684\u306b\u9001\u3063\u3066\u304f\u308c\u308b\u3002<\/p>\n<p>\u6b21\u306b WAL \u3060\u304c\u3001WAL \u306e\u30b3\u30d4\u30fc\u306f\u3001postgresql.conf \u306b\u8a18\u8f09\u3057\u305f archive_command \u306e\u8a2d\u5b9a\u306b\u5f93\u3063\u3066\u884c\u308f\u308c\u308b\u306e\u3067\u3001\u3053\u3053\u306b wal-e \u306e\u30b3\u30de\u30f3\u30c9\u3092\u6307\u5b9a\u3057\u3066\u304a\u304f\u3002<\/p>\n<p>\u30ea\u30b9\u30c8\u30a2\u306b\u95a2\u3057\u3066\u306f\u5f8c\u8ff0\u3002<\/p>\n<p><!--more--><\/p>\n<h2>\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb<\/h2>\n<p>python, lzop, psql, pv \u306b\u4f9d\u5b58\u3057\u3066\u3044\u308b\u306e\u3067\u3001\u305d\u308c\u3089\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3059\u308b\u3002<\/p>\n<p>\u6b21\u306b\u3001pip wal-e \u3067OK\u3002\u3042\u3068\u3001\u3053\u306e\u8a18\u4e8b\u3067\u306f envdir \u3082\u4f7f\u3046\u306e\u3067\u3001\u305d\u308c\u3082 pip \u3067\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u3066\u304a\u304f\u3002<\/p>\n<h2>\u8a2d\u5b9a<\/h2>\n<h3>S3 \u7b49\u306e\u8a8d\u8a3c\u60c5\u5831\u306e\u8a2d\u5b9a<\/h3>\n<p>envdir \u3092\u4f7f\u3046\u3002<span class=\"blob-code-inner\"><span class=\"pl-s\">\/etc\/wal-e.d\/env\/<\/span><\/span> \u306e\u4e0b\u306b\u3001\u8a2d\u5b9a\u3057\u305f\u3044\u74b0\u5883\u5909\u6570\u540d\u3068\u540c\u3058\u540d\u524d\u306e\u30d5\u30a1\u30a4\u30eb\u540d\u3092\u4f5c\u6210\u3057\u3066\u3001\u305d\u3053\u306b\u74b0\u5883\u5909\u6570\u306b\u8a2d\u5b9a\u3057\u305f\u3044\u5024\u3092\u66f8\u304f\u3002\u81ea\u5206\u306e\u5834\u5408\u306f\u3001<a href=\"http:\/\/kazu.tv\/blog\/2015\/06\/30\/openstack-swift-on-vps\/\">\u524d\u56de\u8a2d\u5b9a<\/a>\u3057\u305f OpenStack Swift \u3092\u4f7f\u3046\u306e\u3067\u3001\/etc\/wal-e.d\/env\/ \u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u4ee5\u4e0b\u306b\u3001\u4ee5\u4e0b\u306e\u3088\u3046\u306a\u30d5\u30a1\u30a4\u30eb\u3092\u4f5c\u6210\u3002<\/p>\n<ul>\n<li><span class=\"blob-code-inner\"><span class=\"pl-s\">SWIFT_AUTHURL<\/span><\/span><\/li>\n<li><span class=\"blob-code-inner\"><span class=\"pl-s\">SWIFT_TENANT<\/span><\/span><\/li>\n<li>SWIFT_USER<\/li>\n<li>SWIFT_PASSWORD<\/li>\n<\/ul>\n<h3>\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306e\u8a2d\u5b9a<\/h3>\n<p>postgresql.conf \u3067\u4ee5\u4e0b\u3092\u8a18\u8ff0\u3002<\/p>\n<p>3\u884c\u76ee\u306e\u5185\u5bb9\u304c\u3001WAL \u3092\u30b9\u30c8\u30ec\u30fc\u30b8\u306b\u9001\u308b\u30b3\u30de\u30f3\u30c9\u3068\u306a\u3063\u3066\u3044\u308b\u30024\u884c\u76ee\u306f\u304a\u597d\u307f\u3067\u3001\u9069\u5f53\u306a\u5024\u306b\u3002<\/p>\n<pre class=\"brush: plain;\">wal_level = archive # \u53c8\u306fhot_standby\r\narchive_mode = on\r\narchive_command = '\/path\/to\/envdir \/etc\/wal-e.d\/env \/path\/to\/wal-e wal-push %p'\r\narchive_timeout = 5min\r\n<\/pre>\n<p><span class=\"blob-code-inner\"><span class=\"pl-s\"> \u3042\u3068\u306f\u3001\u30d9\u30fc\u30b9\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306e\u30b3\u30de\u30f3\u30c9\u3092 crontab \u3067\u4ed5\u8fbc\u3093\u3067\u304a\u304f\u3002\u81ea\u5206\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306a\u30b7\u30a7\u30eb\u30b9\u30af\u30ea\u30d7\u30c8\u3092\u4f5c\u3063\u3066\u3001\u305d\u308c\u3092\u65e5\u6b21\u3067\u5b9f\u884c\u3057\u3066\u3044\u308b\u3002<br \/>\n<\/span><\/span><\/p>\n<pre class=\"brush: sh;\">#!\/bin\/bash\r\n\r\nENVDIR_CMD=\/path\/to\/envdir\r\nWALE_CMD=\/path\/to\/wal-e\r\nENVDIR=\/etc\/wal-e.d\/env\r\nDATADIR=\/var\/lib\/postgresql\/9.3\/main\/\r\n\r\n$ENVDIR_CMD $ENVDIR $WALE_CMD backup-push $DATADIR \\\r\n&amp;&amp; $ENVDIR_CMD $ENVDIR $WALE_CMD delete retain 7<\/pre>\n<h3>\u30ea\u30b9\u30c8\u30a2\u306e\u8a2d\u5b9a<\/h3>\n<p>\u3053\u308c\u306e\u3084\u308a\u65b9\u3092\u52d8\u9055\u3044\u3057\u3066\u3044\u3066\u3001\u30c8\u30fc\u30bf\u30eb\u3067\u4e38\u4e00\u65e5\u5206\u640d\u3092\u3057\u305f\u3093\u3060\u3051\u3069\u3001\u6b63\u89e3\u3068\u3057\u3066\u306f\u4ee5\u4e0b\u306e\u5185\u5bb9\u3067 recovery.conf \u3092 PostgreSQL \u306e<strong>\u30c7\u30fc\u30bf\u30c7\u30a3\u30ec\u30af\u30c8\u30ea<\/strong>\u306e\u914d\u4e0b\u306b\u7f6e\u304f\u3002OS\u3001\u30c7\u30a3\u30b9\u30c8\u30ea\u30d3\u30e5\u30fc\u30b7\u30e7\u30f3\u306b\u3088\u3063\u3066\u5dee\u306f\u3042\u308b\u304c\u3001 <span class=\"blob-code-inner\"><span class=\"pl-s\">\/var\/lib\/postgresql\/9.3\/main\/recovery.conf \u3068\u3044\u3046\u30d5\u30a1\u30a4\u30eb\u3092\u4f5c\u308b\u3002<\/span><\/span><\/p>\n<pre class=\"brush: plain;\">restore_command = '\/path\/to\/envdir \/etc\/wal-e.d\/env \/path\/to\/wal-e wal-fetch \"%f\" \"%p\"'<\/pre>\n<p>\u3053\u308c\u3067OK\u3002<\/p>\n<p>\u88dc\u8db3\u3059\u308b\u3068\u3001 recovery.conf \u3068\u306f\u3001PostgreSQL \u306e\u30ea\u30ab\u30d0\u30ea\u306b\u95a2\u3059\u308b\u8a2d\u5b9a\u30d5\u30a1\u30a4\u30eb\u3002\u8a73\u7d30\u306f<a href=\"http:\/\/www.postgresql.org\/docs\/9.4\/static\/recovery-config.html\" target=\"_blank\">\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8<\/a>\u3092\u53c2\u7167\u306e\u3053\u3068\u3002<\/p>\n<h2>\u30ea\u30ab\u30d0\u30ea\u30fb\u30ea\u30b9\u30c8\u30a2<\/h2>\n<p>\u4ee5\u4e0b2\u901a\u308a\u306e\u3044\u305a\u308c\u306e\u5834\u5408\u3082\u3001\u6210\u529f\u3057\u305f\u5834\u5408 recovery.conf \u306f recovery.done \u306b\u30ea\u30cd\u30fc\u30e0\u3055\u308c\u308b\u3002<\/p>\n<h3>\u30af\u30e9\u30c3\u30b7\u30e5\u304b\u3089\u306e\u30ea\u30ab\u30d0\u30ea<\/h3>\n<p>PostgreSQL \u304c\u30af\u30e9\u30c3\u30b7\u30e5\u3057\u305f\u3068\u304d\u306a\u3069\u306f\u3001\u6b21\u56de\u8d77\u52d5\u6642\u306b\u3001recovery.conf \u306e\u5185\u5bb9\u304b\u3089\u81ea\u52d5\u7684\u306b WAL \u3092\u53d6\u5f97\u3057\u3066\u3001WAL \u304c\u3042\u308b\u3068\u3053\u308d\u307e\u3067\u30ea\u30ab\u30d0\u30ea\u3057\u3066\u304f\u308c\u308b\u3002<\/p>\n<h3>0\u304b\u3089\u306e\u30ea\u30b9\u30c8\u30a2<\/h3>\n<p>HDD \u304c\u98db\u3093\u3060\u306a\u3069\u3068\u3044\u3063\u305f\u30c8\u30e9\u30d6\u30eb\u306e\u5834\u5408\u306f\u3001\u307e\u305a\u306f\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3067 S3 \u304b\u3089\u30d9\u30fc\u30b9\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u3092\u53d6\u5f97\u3059\u308b\u3002<\/p>\n<pre class=\"brush: sh;\">\/path\/to\/envdir \/etc\/wal-e.d\/env backup-fetch \/var\/lib\/postgresql\/9.3\/main LATEST<\/pre>\n<p>\u305d\u306e\u5f8c\u3001PostgreSQL \u3092\u8d77\u52d5\u3059\u308c\u3070\u3001\u81ea\u52d5\u7684\u306b WAL \u3092 S3 \u306a\u3069\u304b\u3089\u53d6\u5f97\u3057\u3066\u3001\u30ed\u30b0\u3092\u30ea\u30d7\u30ec\u30a4\u3057\u3066\u304f\u308c\u308b\u3002<\/p>\n<h2>\u307e\u3068\u3081<\/h2>\n<p>pg_dump \u3092\u4f7f\u3046\u7c21\u5358\u306a\u65b9\u6cd5\u3082\u3044\u3044\u3051\u3069\u3001wal-e \u3092\u4f7f\u3046\u3068\u6bd4\u8f03\u7684\u7c21\u5358\u306b PITR \u304c\u51fa\u6765\u308b\u306e\u3067\u30aa\u30b9\u30b9\u30e1\u3002<\/p>\n<p>\u4ee5\u4e0b\u3001\u672c\u5bb6\u4ee5\u5916\u3067\u53c2\u8003\u306b\u3057\u305f\u30b5\u30a4\u30c8\u3002<\/p>\n<ul>\n<li><a href=\"https:\/\/gist.github.com\/ruckus\/2293434\" target=\"_blank\">Basic setup of WAL-E for continuous archiving and recovery<\/a><\/li>\n<li><a href=\"http:\/\/qiita.com\/awakia\/items\/0f36c051a1df8f0b6c6a\" target=\"_blank\">Heroku\u304c\u958b\u767a\u3057\u305fWAL-E\u3092\u4f7f\u3063\u3066\u3001AWS\u3067PostgreSQL\u306e\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u3092\u53d6\u308b<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>PostgreSQL \u306e\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u3092\u3057\u305f\u3044 \u4eca\u81ea\u5206\u305f\u3061\u3067\u4f5c\u3063\u3066\u3044\u308b\u30b5\u30fc\u30d3\u30b9\u3067\u306f\u3001\u6210\u308a\u884c\u304d\u4e0a PostgreSQL \u3092\u4f7f\u3063\u3066\u3044\u308b\u3002\u3067\u3001\u6700\u8fd1\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u306e\u8a2d\u5b9a\u3092\u3057\u305f\u3093\u3060\u3051\u3069\u3001\u4eca\u307e\u3067\u3068\u540c\u3058\u304f pg_dump \u3092\u3069\u3063\u304b\u5225\u306e\u5834\u6240&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[15],"tags":[923,944],"class_list":["post-1556","post","type-post","status-publish","format-standard","hentry","category-15","tag-postgresql","tag-wal-e"],"_links":{"self":[{"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/posts\/1556","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/comments?post=1556"}],"version-history":[{"count":7,"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/posts\/1556\/revisions"}],"predecessor-version":[{"id":1560,"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/posts\/1556\/revisions\/1560"}],"wp:attachment":[{"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/media?parent=1556"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/categories?post=1556"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kazu.tv\/blog\/wp-json\/wp\/v2\/tags?post=1556"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}