mod_pagespeed を FreeBSD で動かす

html や CSS を動的に書き換えて、ページの読み込みを高速化させる Google 製の apache モジュール mod_pagespeed ですが、今のところ FreeBSD はサポートされていません。Web 上にも動かそうとチェレンジしている人が何人かいるようですが、動かしたという話を聞かないので方法を調べてみました。

※この記事を書いた2ヶ月後に、以下に書いたパッチがそのまま FreeBSD の ports ツリーに取り込まれました。今は ports の www/mod_pagespeed をビルドするだけで導入できるようになっています。

調査過程を説明した後、方法を述べます。

コンパイルエラーの対処

Web 上で躓いている人は大抵以下のコンパイルエラーで困っているようです。
  CXX(target) out/Release/obj.target/pagespeed_output_pb/gen/protoc_out/pagespeed/proto/pagespeed_output.pb.
o
In file included from out/Release/obj/gen/protoc_out/pagespeed/proto/pagespeed_output.pb.cc:4:
out/Release/obj/gen/protoc_out/pagespeed/proto/pagespeed_output.pb.h:1470: error: expected unqualified-id before 'int'
調べてみたところ、protocol buffer の定義ファイル pagespeed_output.proto から major/minor というシンボルが生成されており、 これがマクロで定義されている major(3), minor(3) と衝突していました。適当なところで #undef すればエラーを回避できます。

その後は、apr のビルドがエラーになったのでシステムの apr を使うようにすると最後までビルドできました。

テスト失敗の対処

これで動くかなと思いきや、自動テストの pagespeed_automatic_test を実行すると、 FAIL と出た後 abort してしまいます。
これは FreeBSD が pthread の PTHREAD_PROCESS_SHARED をサポートしていないためで、その旨エラーが出力されていました。この部分は pthread_mutex_t ではなく sem_t を使ってプロセス間で排他処理を実現することで対処しました。


これですべてのテストにパスし、正常に apache で mod_pagespeed が動作することが確認できました。


具体的な手順は以下にまとめます。
試した環境は FreeBSD 8.1-RELEASE amd64です。



パッケージの準備

まず以下のパッケージを事前にインストールしておきます。私は ports から入れました。バージョンは違っても問題ないでしょう。
  • apache-2.2.16
  • bash-4.1.10
  • binutils-2.21.1
  • coreutils-8.12
  • gsed-4.2.1_2
  • libexecinfo-1.1_3
  • python26-2.6.7
  • subversion-1.6.12_1

ビルド環境の準備


基本的に公式ドキュメントの HowToBuild で紹介されているとおりですが、いくつかのコマンドが GNU 版と挙動が違うのでその辺のケアをしておきます。

cd
mkdir google
cd google
svn co http://src.chromium.org/svn/trunk/tools/depot_tools
cd depot_tools
ls -s /usr/local/bin/gsed sed
ls -s /usr/local/bin/greadlink readlink
ls -s /usr/local/bin/ginstall install
export PATH=~/google/depot_tools:%PATH%
cd ../
また flock というコマンドが必要になるので、ダミーのスクリプトを depot_tools/flock として用意しておきます。chmod +x しておくのも忘れずに。
#!/bin/sh
shift 1
$*

ソースコードの準備

ソースコードをチェックアウトします。
mkdir mod_pagespeed    # any directory is fine
mod_pagespeed

gclient config http://modpagespeed.googlecode.com/svn/trunk/src
gclient sync --force
cd src
作業したときの svn info は以下でした。
Path: .
URL: http://modpagespeed.googlecode.com/svn/trunk/src
Repository Root: http://modpagespeed.googlecode.com/svn
Repository UUID: b28baae1-b17d-8ef2-060a-10057fd0e0fa
Revision: 866
Node Kind: directory
Schedule: normal
Last Changed Author: jmarantz@google.com
Last Changed Rev: 866
Last Changed Date: 2011-07-29 03:06:58 +0900 (Fri, 29 Jul 2011)

パッチを当てる

以下のパッチを適用します。

ビルドエラー対策
Index: third_party/protobuf/src/google/protobuf/stubs/common.h
===================================================================
--- third_party/protobuf/src/google/protobuf/stubs/common.h     (revision 68180)
+++ third_party/protobuf/src/google/protobuf/stubs/common.h     (working copy)
@@ -47,6 +47,8 @@
 #elif !defined(_MSC_VER)
 #include
 #endif
+#undef major
+#undef minor

 #if defined(_WIN32) && defined(GetMessage)
 // Allow GetMessage to be used as a valid method name in protobuf classes.
Index: third_party/chromium/src/base/debug/stack_trace_posix.cc
===================================================================
--- third_party/chromium/src/base/debug/stack_trace_posix.cc    (revision 68180)
+++ third_party/chromium/src/base/debug/stack_trace_posix.cc    (working copy)
@@ -126,7 +126,7 @@
       trace_strings->push_back(base::StringPrintf("%p", trace[i]));
     }
   }
-#else
+#elif defined(OS_LINUX)
   scoped_ptr_malloc trace_symbols(backtrace_symbols(trace, size));
   if (trace_symbols.get()) {
     for (int i = 0; i < size; ++i) {
@@ -156,9 +156,13 @@
     return;
   }
 #endif
+#if defined(OS_LINUX)
   // Though the backtrace API man page does not list any possible negative
   // return values, we take no chance.
   count_ = std::max(backtrace(trace_, arraysize(trace_)), 0);
+#else
+  count_ = 0;
+#endif
 }

 void StackTrace::PrintBacktrace() {
Index: third_party/chromium/src/base/debug/debugger_posix.cc
===================================================================
--- third_party/chromium/src/base/debug/debugger_posix.cc       (revision 68180)
+++ third_party/chromium/src/base/debug/debugger_posix.cc       (working copy)
@@ -137,7 +137,7 @@

 #elif defined(OS_FREEBSD)

-bool DebugUtil::BeingDebugged() {
+bool BeingDebugged() {
   // TODO(benl): can we determine this under FreeBSD?
   NOTIMPLEMENTED();
   return false;

Index: third_party/icu/public/common/unicode/platform.h
===================================================================
--- third_party/icu/public/common/unicode/platform.h    (revision 0)
+++ third_party/icu/public/common/unicode/platform.h    (revision 0)
@@ -0,0 +1,403 @@
+/*
+******************************************************************************
+*
+*   Copyright (C) 1997-2011, International Business Machines
+*   Corporation and others.  All Rights Reserved.
+*
+******************************************************************************
+*
+* Note: autoconf creates platform.h from platform.h.in at configure time.
+*
+******************************************************************************
+*
+*  FILE NAME : platform.h
+*
+*   Date        Name        Description
+*   05/13/98    nos         Creation (content moved here from ptypes.h).
+*   03/02/99    stephen     Added AS400 support.
+*   03/30/99    stephen     Added Linux support.
+*   04/13/99    stephen     Reworked for autoconf.
+******************************************************************************
+*/
+
+#ifndef _PLATFORM_H
+#define _PLATFORM_H
+
+/**
+ * \file
+ * \brief Basic types for the platform
+ */
+
+/* This file should be included before uvernum.h. */
+#if defined(UVERNUM_H)
+# error Do not include unicode/uvernum.h before #including unicode/platform.h.  Instead of unicode/uvernum.h, #include unicode/uversion.h
+#endif
+
+/**
+ * Determine wheter to enable auto cleanup of libraries.
+ * @internal
+ */
+#ifndef UCLN_NO_AUTO_CLEANUP
+#define UCLN_NO_AUTO_CLEANUP 1
+#endif
+
+#if 0
+#define CYGWINMSVC
+#endif
+
+/* Need platform.h when using CYGWINMSVC to get definitions above. Ignore everything else. */
+#ifndef CYGWINMSVC
+
+/** Define the platform we're on. */
+#ifndef U_BSD
+#define U_BSD
+#endif
+
+/**
+ * \def U_HAVE_DIRENT_H
+ * Define whether dirent.h is available
+ * @internal
+ */
+#ifndef U_HAVE_DIRENT_H
+#define U_HAVE_DIRENT_H 1
+#endif
+
+/** Define whether inttypes.h is available */
+#ifndef U_HAVE_INTTYPES_H
+#define U_HAVE_INTTYPES_H 1
+#endif
+#include
+
+/**
+ * Define what support for C++ streams is available.
+ *     If U_IOSTREAM_SOURCE is set to 199711, then <iostream> is available
+ * (1997711 is the date the ISO/IEC C++ FDIS was published), and then
+ * one should qualify streams using the std namespace in ICU header
+ * files.
+ *     If U_IOSTREAM_SOURCE is set to 198506, then <iostream.h> is
+ * available instead (198506 is the date when Stroustrup published
+ * "An Extensible I/O Facility for C++" at the summer USENIX conference).
+ *     If U_IOSTREAM_SOURCE is 0, then C++ streams are not available and
+ * support for them will be silently suppressed in ICU.
+ *
+ */
+
+#ifndef U_IOSTREAM_SOURCE
+#define U_IOSTREAM_SOURCE 199711
+#endif
+
+/**
+ * \def U_HAVE_STD_STRING
+ * Define whether the standard C++ (STL) <string> header is available.
+ * For platforms that do not use platform.h and do not define this constant
+ * in their platform-specific headers, std_string.h defaults
+ * U_HAVE_STD_STRING to 1.
+ * @internal
+ */
+#ifndef U_HAVE_STD_STRING
+#define U_HAVE_STD_STRING 1
+#endif
+
+/** @{ Determines whether specific types are available */
+#ifndef U_HAVE_INT8_T
+#define U_HAVE_INT8_T 1
+#endif
+
+#ifndef U_HAVE_UINT8_T
+#define U_HAVE_UINT8_T 1
+#endif
+
+#ifndef U_HAVE_INT16_T
+#define U_HAVE_INT16_T 1
+#endif
+
+#ifndef U_HAVE_UINT16_T
+#define U_HAVE_UINT16_T 1
+#endif
+
+#ifndef U_HAVE_INT32_T
+#define U_HAVE_INT32_T 1
+#endif
+
+#ifndef U_HAVE_UINT32_T
+#define U_HAVE_UINT32_T 1
+#endif
+
+#ifndef U_HAVE_INT64_T
+#define U_HAVE_INT64_T 1
+#endif
+
+#ifndef U_HAVE_UINT64_T
+#define U_HAVE_UINT64_T 1
+#endif
+
+/** @} */
+
+/*===========================================================================*/
+/** @{ Compiler and environment features                                     */
+/*===========================================================================*/
+
+/* Define whether namespace is supported */
+#ifndef U_HAVE_NAMESPACE
+#define U_HAVE_NAMESPACE 1
+#endif
+
+/* Determines the endianness of the platform
+   It's done this way in case multiple architectures are being built at once.
+   For example, Darwin supports fat binaries, which can be both PPC and x86 based. */
+#if defined(BYTE_ORDER) && defined(BIG_ENDIAN)
+#define U_IS_BIG_ENDIAN (BYTE_ORDER == BIG_ENDIAN)
+#else
+#define U_IS_BIG_ENDIAN 0
+#endif
+
+/* 1 or 0 to enable or disable threads.  If undefined, default is: enable threads. */
+#ifndef ICU_USE_THREADS
+#define ICU_USE_THREADS 1
+#endif
+
+#ifndef U_DEBUG
+#define U_DEBUG 0
+#endif
+
+#ifndef U_RELEASE
+#define U_RELEASE 1
+#endif
+
+/* Determine whether to disable renaming or not. This overrides the
+   setting in umachine.h which is for all platforms. */
+#ifndef U_DISABLE_RENAMING
+#define U_DISABLE_RENAMING 0
+#endif
+
+/* Determine whether to override new and delete. */
+#ifndef U_OVERRIDE_CXX_ALLOCATION
+#define U_OVERRIDE_CXX_ALLOCATION 1
+#endif
+/* Determine whether to override placement new and delete for STL. */
+#ifndef U_HAVE_PLACEMENT_NEW
+#define U_HAVE_PLACEMENT_NEW 1
+#endif
+
+/* Determine whether to enable tracing. */
+#ifndef U_ENABLE_TRACING
+#define U_ENABLE_TRACING 0
+#endif
+
+/**
+ * Whether to enable Dynamic loading in ICU
+ * @internal
+ */
+#ifndef U_ENABLE_DYLOAD
+#define U_ENABLE_DYLOAD 1
+#endif
+
+/**
+ * Whether to test Dynamic loading as an OS capabilty
+ * @internal
+ */
+#ifndef U_CHECK_DYLOAD
+#define U_CHECK_DYLOAD 1
+#endif
+
+
+/** Do we allow ICU users to use the draft APIs by default? */
+#ifndef U_DEFAULT_SHOW_DRAFT
+#define U_DEFAULT_SHOW_DRAFT 1
+#endif
+
+/** @} */
+
+/*===========================================================================*/
+/** @{ Character data types                                                      */
+/*===========================================================================*/
+
+#if ((defined(OS390) && (!defined(__CHARSET_LIB) || !__CHARSET_LIB))) || defined(OS400)
+#   define U_CHARSET_FAMILY 1
+#endif
+
+/** @} */
+
+/*===========================================================================*/
+/** @{ Information about wchar support                                           */
+/*===========================================================================*/
+
+#ifndef U_HAVE_WCHAR_H
+#define U_HAVE_WCHAR_H      1
+#endif
+
+#ifndef U_SIZEOF_WCHAR_T
+#define U_SIZEOF_WCHAR_T    4
+#endif
+
+#ifndef U_HAVE_WCSCPY
+#define U_HAVE_WCSCPY       1
+#endif
+
+/** @} */
+
+/**
+ * @{
+ * \def U_DECLARE_UTF16
+ * Do not use this macro. Use the UNICODE_STRING or U_STRING_DECL macros
+ * instead.
+ * @internal
+ *
+ * \def U_GNUC_UTF16_STRING
+ * @internal
+ */
+#ifndef U_GNUC_UTF16_STRING
+#define U_GNUC_UTF16_STRING 0
+#endif
+#if 1 || defined(U_CHECK_UTF16_STRING)
+#if (defined(__xlC__) && defined(__IBM_UTF_LITERAL) && U_SIZEOF_WCHAR_T != 2) \
+    || (defined(__HP_aCC) && __HP_aCC >= 035000) \
+    || (defined(__HP_cc) && __HP_cc >= 111106) \
+    || U_GNUC_UTF16_STRING
+#define U_DECLARE_UTF16(string) u ## string
+#elif (defined(__SUNPRO_CC) && __SUNPRO_CC >= 0x550)
+/* || (defined(__SUNPRO_C) && __SUNPRO_C >= 0x580) */
+/* Sun's C compiler has issues with this notation, and it's unreliable. */
+#define U_DECLARE_UTF16(string) U ## string
+#elif U_SIZEOF_WCHAR_T == 2 \
+    && (U_CHARSET_FAMILY == 0 || ((defined(OS390) || defined(OS400)) && defined(__UCS2__)))
+#define U_DECLARE_UTF16(string) L ## string
+#endif
+#endif
+
+/** @} */
+
+/*===========================================================================*/
+/** @{ Information about POSIX support                                           */
+/*===========================================================================*/
+
+#ifndef U_HAVE_NL_LANGINFO_CODESET
+#define U_HAVE_NL_LANGINFO_CODESET  1
+#endif
+
+#ifndef U_NL_LANGINFO_CODESET
+#define U_NL_LANGINFO_CODESET       CODESET
+#endif
+
+#if 1
+#define U_TZSET         tzset
+#endif
+#if 0
+#define U_TIMEZONE
+#endif
+#if 1
+#define U_TZNAME        tzname
+#endif
+
+#define U_HAVE_MMAP     1
+#define U_HAVE_POPEN    1
+
+/** @} */
+
+/*===========================================================================*/
+/** @{ Symbol import-export control                                              */
+/*===========================================================================*/
+
+#ifdef U_STATIC_IMPLEMENTATION
+#define U_EXPORT
+#elif 1
+#define U_EXPORT __attribute__((visibility("default")))
+#elif (defined(__SUNPRO_CC) && __SUNPRO_CC >= 0x550) \
+   || (defined(__SUNPRO_C) && __SUNPRO_C >= 0x550)
+#define U_EXPORT __global
+/*#elif defined(__HP_aCC) || defined(__HP_cc)
+#define U_EXPORT __declspec(dllexport)*/
+#else
+#define U_EXPORT
+#endif
+
+/* U_CALLCONV is releated to U_EXPORT2 */
+#define U_EXPORT2
+
+/* cygwin needs to export/import data */
+#if defined(U_CYGWIN) && !defined(__GNUC__)
+#define U_IMPORT __declspec(dllimport)
+#else
+#define U_IMPORT
+#endif
+
+/* @} */
+
+/*===========================================================================*/
+/** @{ Code alignment and C function inlining                                    */
+/*===========================================================================*/
+
+#ifndef U_INLINE
+#   ifdef __cplusplus
+#       define U_INLINE inline
+#   else
+#       define U_INLINE __inline__
+#   endif
+#endif
+
+#ifndef U_ALIGN_CODE
+#define U_ALIGN_CODE(n)
+#endif
+
+/** @} */
+
+/*===========================================================================*/
+/** @{ GCC built in functions for atomic memory operations                       */
+/*===========================================================================*/
+
+/**
+ * \def U_HAVE_GCC_ATOMICS
+ * @internal
+ */
+#ifndef U_HAVE_GCC_ATOMICS
+#define U_HAVE_GCC_ATOMICS 1
+#endif
+
+/** @} */
+
+/*===========================================================================*/
+/** @{ Programs used by ICU code                                                 */
+/*===========================================================================*/
+
+/**
+ * \def U_MAKE
+ * What program to execute to run 'make'
+ */
+#ifndef U_MAKE
+#define U_MAKE  "/usr/local/bin//gmake"
+#endif
+
+/** @} */
+
+#endif /* CYGWINMSVC */
+
+/*===========================================================================*/
+/* Custom icu entry point renaming                                                  */
+/*===========================================================================*/
+
+/**
+ * Define the library suffix with C syntax.
+ * @internal
+ */
+# define U_LIB_SUFFIX_C_NAME
+/**
+ * Define the library suffix as a string with C syntax
+ * @internal
+ */
+# define U_LIB_SUFFIX_C_NAME_STRING ""
+/**
+ * 1 if a custom library suffix is set
+ * @internal
+ */
+# define U_HAVE_LIB_SUFFIX 0
+
+#if U_HAVE_LIB_SUFFIX
+# ifndef U_ICU_ENTRY_POINT_RENAME
+/* Renaming pattern:    u_strcpy_41_suffix */
+#  define U_ICU_ENTRY_POINT_RENAME(x)    x ## _ ## 48 ##
+#  define U_DEF_ICUDATA_ENTRY_POINT(major, minor) icudt####major##minor##_dat
+
+# endif
+#endif
+
+#endif
テスト失敗対策
Index: net/instaweb/util/pthread_shared_mem.cc
===================================================================
--- net/instaweb/util/pthread_shared_mem.cc (revision 866)
+++ net/instaweb/util/pthread_shared_mem.cc (working copy)
@@ -24,6 +24,7 @@
 #include <cstddef>
 #include <map>
 #include <utility>
+#include <semaphore.h>
 #include "net/instaweb/util/public/abstract_shared_mem.h"
 #include "net/instaweb/util/public/abstract_mutex.h"
 #include "net/instaweb/util/public/basictypes.h"
@@ -48,7 +49,6 @@
     }
   }
 }
-
 // Unlike PthreadMutex this doesn't own the lock, but rather refers to an
 // external one.
 class PthreadSharedMemMutex : public AbstractMutex {
@@ -66,16 +66,38 @@

  private:
   pthread_mutex_t* external_mutex_;
-
   DISALLOW_COPY_AND_ASSIGN(PthreadSharedMemMutex);
 };

+
+// Some OSs does not support PTHREAD_PROCESS_SHARED attribute. (ex. FreeBSD < 9R, MacOS)
+// So, we use the semaphore instead. But nested lock is not supported.
+class SemSharedMemMutex : public AbstractMutex {
+ public:
+  explicit SemSharedMemMutex(sem_t* sem)
+      : sem_(sem) {}
+
+  virtual void Lock() {
+    sem_wait(sem_);
+  }
+
+  virtual void Unlock() {
+    sem_post(sem_);
+  }
+
+private:
+  sem_t* sem_;
+  DISALLOW_COPY_AND_ASSIGN(SemSharedMemMutex);
+};
+
 class PthreadSharedMemSegment : public AbstractSharedMemSegment {
  public:
   // We will be representing memory mapped in the [base, base + size) range.
-  PthreadSharedMemSegment(char* base, size_t size, MessageHandler* handler)
+  PthreadSharedMemSegment(char* base, size_t size, MessageHandler* handler, const char* name)
       : base_(base),
-        size_(size) {
+        size_(size),
+        sem_(NULL) {
+    std::sprintf(name_, "/tmp/.%s_sem", name);
   }

   virtual ~PthreadSharedMemSegment() {
@@ -86,10 +108,15 @@
   }

   virtual size_t SharedMutexSize() const {
+#if 0
     return sizeof(pthread_mutex_t);
+#else
+    return 0;
+#endif
   }

   virtual bool InitializeSharedMutex(size_t offset, MessageHandler* handler) {
+#if 0
     pthread_mutexattr_t attr;
     if (pthread_mutexattr_init(&attr) != 0) {
       handler->Message(kError, "pthread_mutexattr_init failed with errno:%d",
@@ -113,20 +140,33 @@

     pthread_mutexattr_destroy(&attr);
     return true;
+#else
+//std::printf("sem: %s\n", name_);
+    sem_ = sem_open(name_, O_CREAT, 0755, 1);
+    return sem_ ? true : false;
+#endif
   }

   virtual AbstractMutex* AttachToSharedMutex(size_t offset) {
+#if 0
     return new PthreadSharedMemMutex(MutexPtr(offset));
+#else
+    sem_ = sem_open(name_, O_EXCL);
+    return new SemSharedMemMutex(sem_);
+#endif
   }

  private:
+#if 0
   pthread_mutex_t* MutexPtr(size_t offset) {
     return reinterpret_cast(base_ + offset);
   }
+#endif

   char* const base_;
   const size_t size_;
-
+  char name_[256];
+  sem_t* sem_;
   DISALLOW_COPY_AND_ASSIGN(PthreadSharedMemSegment);
 };

@@ -143,7 +183,11 @@
 }

 size_t PthreadSharedMem::SharedMutexSize() const {
+#if 0
   return sizeof(pthread_mutex_t);
+#else
+  return 0;
+#endif
 }

 AbstractSharedMemSegment* PthreadSharedMem::CreateSegment(
@@ -167,7 +211,7 @@
   SegmentBaseMap* bases = AcquireSegmentBases();
   (*bases)[name] = base;
   UnlockSegmentBases();
-  return new PthreadSharedMemSegment(base, size, handler);
+  return new PthreadSharedMemSegment(base, size, handler, name.c_str());
 }

 AbstractSharedMemSegment* PthreadSharedMem::AttachToSegment(
@@ -182,7 +226,7 @@
   }
   char* base = i->second;
   UnlockSegmentBases();
-  return new PthreadSharedMemSegment(base, size, handler);
+  return new PthreadSharedMemSegment(base, size, handler, name.c_str());
 }

 void PthreadSharedMem::DestroySegment(const GoogleString& name,

ビルド

システムの apr を使用するように指定した後通常通りビルドします。
GYP_DEFINES="use_system_apache_dev=1 system_include_path_httpd=/usr/local/include/apache22 system_include_path_apr=/usr/local/include/apr-1 system_include_path_aprutil=/usr/local/include/apr-1" gclient runhooks

gmake BUILDTYPE=Release LDFLAGS=-L/usr/local/lib

インストール

su
cd install
APXS_BIN=/usr/local/sbin/apxs NO_SUDO=1 bash install_apxs.sh


これらの手順はおそらく MacOS X, OpenIndiana など他の OS でも使えるはずです。

コメント

このブログの人気の投稿

国産キャリア版Xperia 1 (802SO) のbootloaderをunlockしてrootを取得する

Synology ds918+ に 2.5Gbps / 5Gbps の USB LAN を接続する

Xperia 1 (802SO) のrootを何も失わずに取得する