Tuesday 21 May 2013

gem install mysql2 missing mysql.h on OS X

Ok. The problem surfaced after I needed Ben to help on a site I was building.
He didn't have ruby or mysql installed on his laptop.
The site was built using Ruby-2.0.0-p195 and Rails-4.0.0-rc1.
That, in itself, isn't the issue.

We went and installed the x86_64 version of MySQL 5.6 on his machine.
This sets up in /usr/local/mysql.
Got a bit annoyed as the install doesn't have any StartupItems so start/stop is manual.
Ce la vie.

I now installed Ruby-2.0.0-p195 with no dramas.

The site code was loaded up into a folder and "bundle install" was run.
That's when the fun started.
The mysql2 gem wouldn't compile.
You get something like this:

This could take a while...
ERROR:  Error installing mysql2:
 ERROR: Failed to build gem native extension.

    /usr/local/rvm/rubies/ruby-2.0.0-p195/bin/ruby extconf.rb
checking for rb_thread_blocking_region()... yes
checking for rb_wait_for_single_fd()... yes
checking for mysql.h... no
checking for mysql/mysql.h... no
-----
mysql.h is missing.  please check your installation of mysql and try again.
-----
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
 --with-opt-dir
 --with-opt-include
 --without-opt-include=${opt-dir}/include
 --with-opt-lib
 --without-opt-lib=${opt-dir}/lib
 --with-make-prog
 --without-make-prog
 --srcdir=.
 --curdir
 --ruby=/usr/local/rvm/rubies/ruby-2.0.0-p195/bin/ruby
 --with-mysql-config
 --without-mysql-config


Gem files will remain installed in /usr/local/rvm/gems/ruby-2.0.0-p195/gems/mysql2-0.3.11 for inspection.
Results logged to /usr/local/rvm/gems/ruby-2.0.0-p195/gems/mysql2-0.3.11/ext/mysql2/gem_make.out

Wait. What? The mysql.h does exist (and is valid) in the /usr/local/mysql/include folder.
I, of course, realised that I had not set the --with-mysql-include folder and tried that along with some others.
Nope.
No luck.

I did some googling and the options where confusing and ultimately useless.
The most common options are:
  • Install mysql server using homebrew. Wait. What?
  • Setting env ARCHFLAGS="-arch x86_64" before running gem.
  • Using bundle to create a .bundle/config file.
  • Including every --with-mysql option available on the gem install.
I didn't reinstall mysql as that seemed like silly talk.
I did try the --with-mysql-config=/usr/local/mysql/bin/mysql_config option with no luck.
I had this tickling sensation in my brain that indicated that I was buggerising around in the wrong 'mode' and that I was missing something gob smackingly obvious.
But I persevered.

I tested the /usr/local/mysql/bin/mysql_config program and it's output was like this:

Usage: /usr/local/mysql/bin/mysql_config [OPTIONS]
Options:
        --cflags         [-I/usr/local/mysql/include  -Wno-unused-private-field -Os -g -fno-strict-aliasing -arch x86_64]
        --cxxflags       [-I/usr/local/mysql/include  -Wno-unused-private-field -Os -g -fno-strict-aliasing -arch x86_64]
        --include        [-I/usr/local/mysql/include]
        --libs           [-L/usr/local/mysql/lib -lmysqlclient]
        --libs_r         [-L/usr/local/mysql/lib -lmysqlclient_r]
        --plugindir      [/usr/local/mysql/lib/plugin]
        --socket         [/tmp/mysql.sock]
        --port           [0]
        --version        [5.6.11]
        --libmysqld-libs [-L/usr/local/mysql/lib -lmysqld]
        --variable=VAR   VAR is one of:
                pkgincludedir [/usr/local/mysql/include]
                pkglibdir     [/usr/local/mysql/lib]
                plugindir     [/usr/local/mysql/lib/plugin]

So I did some digging around in the gems folder and under the ext/mysql2 folder there was the mkmf.log file.
As I was stuffed up with flu, surrounded by used tissues and it was 3am it took me a little while to see the issue.
Then I saw it:

cc1: error: unrecognized command line option "-Wno-null-conversion"

It's cc1.
It's failing because it has been provided with a -W gcc option that isn't supported by the compiler.
I tried without luck to get "gem" and the "extconf.rb" to use the ENV['CC'] variable.
So using the gcc-4.2 compiler was the only option.
I checked the compiler version:

gcc -v
Using built-in specs.
Target: i686-apple-darwin11
Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~25/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~25/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)

Then it struck me that I was an idiot.
I edited the /usr/local/mysql/bin/mysql_config file and removed the offending -W compiler options.
(It's around line 120 or so)
The original looks like this:

cflags="-I$pkgincludedir  -Wall -Wno-null-conversion -Wno-unused-private-field -Os -g -fno-strict-aliasing -DDBUG_OFF -arch x86_64 " #note: end space!
cxxflags="-I$pkgincludedir  -Wall -Wno-null-conversion -Wno-unused-private-field -Os -g -fno-strict-aliasing -DDBUG_OFF -arch x86_64 " #note: end space!

And the changed version:

cflags="-I$pkgincludedir  -Wall -Os -g -fno-strict-aliasing -DDBUG_OFF -arch x86_64 " #note: end space!
cxxflags="-I$pkgincludedir  -Wall -Os -g -fno-strict-aliasing -DDBUG_OFF -arch x86_64 " #note: end space!

Now the install works just peachy.

Oh. One last point.
I had to link the dylib to /usr/local/lib:

ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/lib/libmysqlclient.18.dylib











29 comments:

  1. Ugh, thanks so much for posting this - I too was led down many empty alleyways until I stumbled on your post....

    ReplyDelete
  2. Hey Kimbo, thank you sooooo much for the solution! I was having the same problem here and this helped me.

    ps.: sorry for my bad english, and thank you again.

    ReplyDelete
  3. This post of yours saved my day.
    None of the solutions available on Google worked as smooth as this one did.
    Thank you for digging into this issue and making the solution available.

    ReplyDelete
  4. This is awesome. Thanks a lot!

    ReplyDelete
  5. WINNER! Thanks.

    ReplyDelete
  6. Thanks. That works fine and saved me so much time!

    ReplyDelete
  7. Great solution and description. Thank you very much!

    ReplyDelete
  8. Thanks it
    works like charme :)

    ReplyDelete
  9. Hi I looked at my mysql_config and here is what it looks like on line 119

    cflags="-I$pkgincludedir -Wall -Os -g -fno-strict-aliasing -DDBUG_OFF -arch x86_64 " #note: end space!
    cxxflags="-I$pkgincludedir -Os -g -fno-strict-aliasing -DDBUG_OFF -arch x86_64 " #note: end space!
    include="-I$pkgincludedir"

    When I changed it to your configuration it did not solve the issue. Any ideas?

    ReplyDelete
  10. Hi Detmar. Can you give more information about what isn't working? Perhaps you could create a gist (and link to it here) with the output from the gem_make.out file so we can see what actually failed?

    ReplyDelete
  11. Thanks, You saved my day!!!

    ReplyDelete
  12. you are a Genius !!!!!!!!!!!!!!!! , really thank you pal

    ReplyDelete
  13. Just came here to say this worked great! Thanks for the writeup!

    ReplyDelete
  14. thank you so much for sharing! Never thought 'upgrading' to a new MBAir would leave me stranded like a whale on the shores of installation :)

    ReplyDelete
  15. thanks ! I found this post after I saw it on stackoverflow, helped me a bunch !

    ReplyDelete
  16. aaaaaaaaaaaaaaaa
    aaaaa

    aaa

    a
    a
    a

    a

    a

    dude... now *this* is a proper explanation of how to solve this stupid problem... if only i had seen this six hours ago...

    thanks!

    ReplyDelete
  17. Thank you so much! I spent 5 hours without any success and your solution worked like a charm!

    ReplyDelete
  18. Thank you so much ... this helped me a lot. I upgraded to Mac mountain lion and decided to upgrade to ruby 2.0 from 1.9.3 and things just wont work. Initially I thought it was my OSX, then somebody asked to roll back to 1.9.3, then all the usual stuff ..... this solution finally worked.

    ReplyDelete
  19. awesome ! Thank you !

    ReplyDelete
  20. Thanks so much for this! This was really helpful!

    ReplyDelete
  21. This is really awesome!! thanks for the help!!

    ReplyDelete