Tuesday, February 16, 2016

Loading JSON into MariaDB or even MySQL - mysqljsonimport 2.0 is available

It was a long time since I updated mysqljsonimport or mysqljsonexport and I had a few things I wanted to do with them. This release is significant enough for me to bump it up to 2.0, and the same is in the works for mysqljsonexport. The one big thing that is now implemented is reasonably advanced support for MariaDB Dynamic Columns, and it is actually pretty flexible, allowing you to load a nested JSON object into a MariaDB Dynamic Column. But don't worry, it will still link and run with MySQL if that is what you want to do (but then you will not have the dynamic column features, for obvious reasons),

Download from Sourceforge as usual (yes, I know I am oldfashioned and that I should have put it on github). Also as usual is the documentation in pdf format that is also downloadable separately.


Monday, December 14, 2015

Getting started with MariaDB on IBM POWER 8

IBM POWER 8 is latest generation of the IBM POWER series, and it's a hot one. Above all, for you reading this, POWER 8 is the most Linux friendly so far and IBM really wants you to try this out. Seveal Linux distributions are supporting POWER 8 now, and MariaDB is of course the database of choise. Some cools things with the POWER 8 architecture are the support for CAPI (google for more details) and the fact that POWER 8 machines, due to a vastly superior memory architecture, can grow in memory size, which in general is good news but if you want your own POWER 8, this makes then a bit expensive (although maybe not when you consider the performance you get). IBM has fixed that recently and have announced the LC series of servers which start at $6.600 (see more here: http://www-03.ibm.com/systems/power/hardware/linux-lc.html).

So, whar about MariaDB then? Well, MariaDB is a standard component in the Linux distributions that support IBM POWER 8, but we here at MariaDB didn't stop there. We have made numerous fixes to MariaDB to make it perfom at it's best and to increase stability even more. To get at those nice additions and enhancements though, you have to run with the latest MariaDB versions and use the binary builds we provide, and here I'll show you how to do that.

To begin with, you have to register with MariaDB.com, which is free if you want to try MariaDB Enterprise. So surf to MariaDB.com and you should get something like this:
On the top right, as indicated above, are "Login" and "Sign up" links. Select the last of these two and follow the procedure. Then you can revsit this page and log in, and the the links at the top right now say "Logout", "My Portal" and "Profile". Click on the "My Portal" link and you get to a page that looks like this:
Above is indicated the "Downloads" tab, click on this and you will be taken to a page with many different download sections. In this case, let's assume you are on Ubuntu, then select this:
Now it is time to do the actual installation, just follow the steps listed on the page, as I write this, this means I will run:
wget https://downloads.mariadb.com/enterprise/dnae-wefq/generate/10.0/mariadb-enterprise-repository.deb
 dpkg -i mariadb-enterprise-repository.deb
 sudo apt-get update
sudo apt-get install mariadb-server
Following all this. MariaDB is now up and running, but I see you asking yourself, what about POWER 8? How do I install MariaDB on that? And fact is, that is exactly what we have dne here. Or Intel x86 for that matter, the procedure is exactky the same, the differences are handles behind the scenes.

Happy power hacking, I will get back soon with a writeup on MaxScale on POWER 8

Wednesday, August 26, 2015

MyOraDump, Oracle dump utility, version 1.2

I have now released version 1.2 of MyOraDump, my Oracle data extraction tool. This version has one new feature, which is transaction support for MySQL format exports which does speed up loading data a lot! Trust me, it really does! Also I have fixed a bug that did cause a crash at the end of the run, I have no idea why this didn't show up before, but there you go and now it is fixed.

MyOraDump 1.2 can be downloaded sourceforge, and as usual there is also a pdf only download if you want to read up on the tool before using it.


Friday, August 7, 2015

Oracle dump utility version 1.1

Today I released version 1.1 of myoradump for download from sourceforge. If you don't know what myoradump is, this is a utility for exporting data from an Oracle database in some relevant text format so that it can be imported to some other database.

The main thing in version 1.1 is that I have added a whole bunch of new output formats, so make it even easier to get your data out of expensive Oracle and into something more effective. The new formats supported are:
  • MySQL - The format of this is a bunch of INSERT statements that you get when you use mysqldump for example and is useful for import into MariaDB (and MySQL). INSERT arrays are supported as a bunch of more options.
  • JSON - This format is rather obvious, the output is a file consisting of one JSON object per row. To support binary data, which is a no-no in JSON, base64 encoding of binary data is also supported.
  • JSON Array - The format is similar to JSON, but instead of separate objects per row, this format consists of one or more JSON arrays of JSON objects.
  • HTML - This format will produce a valid HTML TABLE. This is sometimes useful when you want to view output data that includes UTF8 characters for example.
In additions, this version of  myoradump includes a bunch of new features and bug fixes. I will follow up this post with one that includes some specific examples of using myoradump eventually.

So, don't touch that dial!

Friday, June 26, 2015

Oracle dump utility

I have created a small program to extract data from an Oracle database in a format suitable for importing into MariaDB (and MySQL I guess). It is pretty fast as it is written in C and uses the OCI interface. Also, it supports most of the Oracle basic types, including BLOB, CLOB and LONG. UTF8 is also supported and there are many generic features, as well as some features specific for later import into MariaDB.

Download the 1.0 version from Sourceforge where the programs source, that is using autotools for building, as well as documentation is available. I have not tested to build on any other version of Oracle than 11, but maybe someone could help me there.


MariaDB with Galera available on the IBM Power8 platform

It was a very long time since I wrote something in this blob, but I have been very busy this spring with MariaDB on Power mostly. This has been a lot of work, but also a lot of fun. So, what is this MariaDB on Power thing all about, well I wrote an introduction to the Power platform late last year. Since then a lot of things has happened though.

One thing is that several service providers out there has adopted Power8 as a platform. To be honest, this really isn't sexy, but it is useful and as a user of one of these services, you will just see the same old Linux you are used to, but potentially it is more powerful and reliable. One such provider is OVH, whose service is more known as RunAbove. If you want to try it, you can do so for free for 7 7 days, just go there and off you go.

Another important thing is that MariaDB is now available on Power8 running, RedHat, SUSE or Ubuntu Linux. To get access to this, pop by MariaDB and if you are not yet signed up, then do this now and then go to "My Portal", further to "Downloads" and then select "MariaDB Enterprise and MariaDB Enterprise Cluster". You are now ready to install using the operating system of your choise, but on Power you are, as I said before,limited to SUSE, RedHat and Ubuntu, and if you want to test MariaDB Enterprise Cluster, i.e. MariaDB with Galera, you have to go with Ubuntu.

Installing MariaDB Enterprise Cluster on Power8 is no more complex than on Intel. There are a few thing to adjust before you can get started with this, after having installed the software. The first node has, as usual, to be configured with wsrep_cluster_adress set to gcomm:// to ensure that this first node will bootstrap without having to connect to a cluster. Once the cluster is up and running though, this variable is set to the cluster addresses. In my case, this what the Galera setting look like in /etc/mysql/my.cnf which is the location of this file on Ubuntu.
# Galera

Note in particular the binlog_format setting. This MUST be set to ROW for Galera to work. But fact is that these setting are not particular to MariaDB on Power, this is the same even on Intel.

Tf this isn't enough to convice you about the advantages of running MariaDB on IBM Power, then see what Foedus in Italy has to say about this combination in this Video:

There is more to say about running MariaDB on Power and there is more to come here, I'll look at some performance data, we'll have a look at MaxScale on Power (this is not official yet, but that isn't stopping me) as well as a blog on how to run a Power8 emulation on Intel which I have promissed before.

So, don't touch that dial!


Tuesday, December 16, 2014

If you can't do it with Power, you can do it with MORE Power

IBM has released their Power8 CPU and with that a bunch of servers and also some other news around this, like the OpenPower consortium that has been around for a year or so now. Besides IBM, Tyan and Google have announced that they are building systems around the Power8 CPU, with the a Tyan server being on sale for some $2800, but this is not a server with a high-end configuration, but it is in a 2U rack mount case and is extensible with more disks and RAM.

There is one thing with the Power8 CPU that sets it apart from the previous Power CPUs that is worth mentioning, which is the byte ordering or endianess of the CPU. The order of the bytes in value in a computer comes in two main flavours, the "Motorola style" AKA "big endian" which is how we write numbers in general, with the most significant digit (or byte in the case of computers) first, and then we have the "Intel style" or "Little endian" where the most significant byte comes last. All of this has little meaning to most of you so far. Thar Big endian is old school mainframe style, but is also used by many RISC CPUs. For those of you with a flair for weird facts one oddball of endianess was PDP-11, where each byte in a 16-bit part in a 32-bit word was swapped (this is called PDP-endian). And by the way, Oddball of Endianess would be a great name for a Country-Rock band (having at least 3 Telecaster equipped guitarists on stage).

What makes Power8 a bit different is that it supports operating systems using either big endian or little endian. Big endian is used by AIX and some Linuxes, but IBM is trying to get more Linuxes onto the little endian train. Among the Big endian Linuxes on Power8 are Red Hat and derivatives, like Fedora (note that CentOS isn't one of them, so far, as Centos 7 doesn't yet support Power at all, nor is 32-bit Intel supported anymore) as well as Debian and SUSE 11 (which will continue to be supported). The little endian Linuxes on Power8 so far includes SUSE 12 and Ubuntu 14.04.  But IBM really wants Linux on Power to run Little endian, so it seems that Debian is on it's way there and Red Hat 7.1 will also support Little endian. But to be honest, we don't have the full story yet.

There are some really good things with the Power8 architecture, like really good performance (see this one for example by MariaDB and IBM) and also it is a great platform for Virtualization / Cloud infrastructure. And all that fuzz with endianess will hopefully be history soon when we have all Linuxes on Power8, as well as x86_64 of course, running Little endian (thank you). And the Country-Rock band mentioned above could call their first record "Fuzz with Endianess".

So, I have convinced you? If so, I guess you want to know what it costs. Well, Power8 machines from  IBM starts at around $10.000, which is a lot it seems, but then you have to remember that used the way this type of system is probably mostly used, as a host for numerous virtual machines, then it is actually pretty reasonable. If you really want a less expensive machine, Tyan has one available which should sell for some $3.000 which is looking more reasonable. Eventually I guess Tyan will sell their motherboards, and the components in this, beside the CPU, is pretty standard stuff (DDR RAM, PCI buses, SATA disks etc). But the CPU is pretty expensive. Anyway, there is a way around all this, if you really want to try running Power8, and that is to use emulation, and once you get the hang of it, this works well, although slowly, but no so slow that you cannot test things on it (but too slow for production use I guess).

I'll show you how to set up an emulated Power8 system in a later blog post, so stay tuned and Don't touch that dial!