Planet JM

Blog

  • The ManageIQ Design Summit – a small intimate gathering of cloud experts

    We’re happy to announce the preliminary agenda for the upcoming ManageIQ Design Summit, a 2-day event on October 7 & 8 in Montvale, NJ. Be sure to RSVP soon, as space is very limited. As mentioned in the title, it’s a small intimate gathering of cloud experts, those interested in pushing the limits of ManageIQ and setting the roadmap for development. If you’re a ManageIQ user who wants to learn how to make the most of its automation and orchestration capabilities, then there will be plenty for you, too:

    • Tour the new RESTful APIs released in Anand
    • Create reusable components for automation and orchestration of your hybrid cloud infrastructure
    • Hack rooms for those who want to dive in

    The proud sponsors of the event are Red Hat and Booz Allen Hamilton. I’ve been told to be on the lookout for a new open source cloud broker project from the Booz Allen engineers.

    Look forward to seeing you there!

  • Moving on From Gluster

    All good things must come to an end. I can say with no equivocation that the last three years have been the most rewarding from a work perspective than any other job I’ve ever had. When I accepted this challenge in May, 2011, I had no idea that the project and community would blossom as they have. I had no idea how many great people were already in place to push this project to the forefront of open source development. I had no idea how many great partners we would find who share our vision for open source storage. I also, of course, didn’t know that Gluster, Inc. would be acquired within months of my arrival, which drastically increased the velocity of the project and community. I didn’t know any of that – what I did know was that there was a pretty cool project called GlusterFS and it seemed like the way forward for storage.

    After we were acquired, we knew there would be a bit of angst from the community about whether we would still actively support other distributions outside of the Red Hat arena. I’m proud to say that we have done that, with active contributions from various community members for Ubuntu, Debian, NetBSD and OpenSUSE builds. We always strove to make gluster.org a truly open community and, in some respects, “bigger than Red Hat.”

    Along the way, we created a board consisting of active community members and organizations. We made the project more transparent and active than ever. We greatly increased the degree that the community is a collaborative experience beyond just the immediate development team. And we greatly increased the reach and scope of the open source storage ecosystem. I can look back and feel privileged to have worked with such amazing visionaries, developers and community evangelists.

    Now it’s time to turn the Gluster community over to someone who can build on what we’ve done and take it even further. I’m staying at Red Hat but moving on to other projects and communities. The ideal candidate should know their way around open source projects and communities, should have an unyielding desire to push things beyond the status quo, should know a thing or two about business strategy, and should understand how to identify which organizations should invest in a community and sell them on the vision. As I’ve mentioned before, today’s community leaders are the equivalent of startup executives, having to mesh together product management and marketing, business development and strategy, sales and messaging into a cohesive whole.

    Are you the next Gluster Community Leader? Drop me a line on IRC – I’m “johnmark” on the Freenode network.

  • The Rise of Open Source Analytics Software

    I was pleased to read about the progress of Graylog2, ElasticSearch, Kibana, et al. in the past year. Machine data analysis has been a growing area of interest for some time now, as traditional monitoring and systems management tools aren’t capable of keeping up with All of the Things that make up many modern workloads. And then there are the more general purpose, “big data” platforms like Hadoop along with the new in-memory upstarts sprouting up around the BDAS stack. Right now is a great time to be a data analytics person, because there has never in the history of computing been a richer set of open source tools to work with.

    There’s a functional difference between what I call data processing platforms, such as Hadoop and BDAS, and data search presentation layers, such as what you find with the ELK stack (ElasticSearch, Logstash and Kibana). While Hadoop, BDAS, et al. are great for processing extremely large data sets, they’re mostly useful as platforms for people Who Know What They’re Doing (TM), ie. math and science PhDs and analytics groups within larger companies. But really, the search and presentation layers are, to me, where the interesting work is taking place these days: it’s where Joe and Jane programmer and operations person are going to make their mark on their organization. And many of the modern tools for data presentation can take data sets from a number of sources: log data, JSON, various forms of XML, event data piped directly over sockets or some other forwarding mechanism. This is why there’s a burgeoning market around tools that integrate with Hadoop and other platforms.

    There’s one aspect of data search presentation layers that has largely gone unmentioned. Everyone tends to focus on the software, and if it’s open source, that gets a strong mention. No one, however, seems to focus on the parts that are most important: data formats, data availability and data reuse. The best part about open source analytics tools is that, by definition, the data outputs must also be openly defined and available for consumption by other tools and platforms. This is in stark contrast to traditional systems management tools and even some modern ones. The most exciting premise of open source tooling in this area is the freedom from the dreaded data roach motel model, where data goes in, but it doesn’t come out unless you pay for the privilege of accessing the data you already own. Recently, I’ve taken to calling it the “skunky data model” and imploring people to “de-skunk their data.”

    Last year, the Red Hat Storage folks came up with the tag line of “Liberate Your Information.” Yes, I know, it sounds hokey and like marketing double-speak, but the concept is very real. There are, today, many users, developers and customers trapped in the data roach motel and cannot get out, because they made the (poor) decision to go with a vendor that didn’t have their needs in mind. It would seem that the best way to prevent this outcome is to go with an open source solution, because again, by definition, it is impossible to create an open source solution that creates proprietary data – because the source is open to the world, it would be impossible to hide how the data is indexed, managed, and stored.

    In the past, one of the problems is that there simply weren’t a whole lot of choices for would-be customers. Luckily, we now have a wealth of options to choose from. As always, I recommend that those looking for solutions in this area go with a vendor that has their interests at heart. Go with a vendor that will allow you to access your data on your terms. Go with a vendor that gives you the means to fire them if they’re not a good partner for you. I think it’s no exaggeration to say that the only way to guarantee this freedom is to go with an open source solution.

    Further reading:

     

  • Citrix and Harvard FASRC Join Gluster Community; Board Expands

    Citrix, Harvard University FASRC and long-time contributors join the Gluster Community Board to drive the direction of open software-defined storage

    February 5, 2014 – The Gluster Community, the leading community for open software-defined storage, announced today two new organizations have signed letters of intent to join: Citrix, Inc. and Harvard University’s Faculty of Arts and Science Research Computing (FASRC) group. This marks the third major expansion of the Gluster Community in governance and projects since mid-2013. Monthly downloads of GlusterFS have tripled since the beginning of 2013, and traffic to gluster.org has increased by over 50% over the previous year. There are now 45 projects on the Gluster Forge and more than 200 developers, with integrations either completed or in the works for Swift, CloudStack, OpenStack Cinder, Ganeti, Archipelago, Xen, QEMU/KVM, Ganesha, the Java platform, and SAMBA, with more to come in 2014.

    Citrix and FASRC will be represented by Mark Hinkle, Senior Director of Open Source Solutions, and James Cuff, Assistant Dean for Research Computing, respectively, joining two individual contributors: Anond Avati, Lead GlusterFS Architect, and Theron Conrey, a contributing speaker, blogger and leading advocate for converged infrastructure. Rounding out the Gluster Community Board are Xavier Hernandez (DataLab); Marc Holmes (Hortonworks), Vin Sharma (Intel), Jim Zemlin (The Linux Foundation), Keisuke Takahashi (NTTPC), Lance Albertson (The Open Source Lab at Oregon State University), John Mark Walker (Red Hat), Louis Zuckerman, Joe Julian, and David Nalley.

    Citrix

    Citrix has become a major innovator in the cloud and virtualization markets. They will drive ongoing efforts to integrate GlusterFS with CloudStack (https://forge.gluster.org/cloudstack-gluster) and the Xen hypervisor. Citrix is also sponsoring Gluster Community events, including a Gluster Cloud Night at their facility in Santa Clara, California on March 18.

    Harvard FASRC

    The research computing group at Harvard has one of the largest known deployments of GlusterFS in the world, pushing GlusterFS beyond previously established limits. Their involvement in testing and development has been invaluable for advancing the usability and stability of GlusterFS.

    Anand Avati

    Anand Avati was employee number 3 at Gluster, Inc. in 2007 and has been the most prolific contributor to the GlusterFS code base as well as its most significant architect over the years. He is primarily responsible for setting the roadmap for the GlusterFS project. Avati is employed by Red Hat but is an individual contributor for the board.

    Theron Conrey

    Theron became involved in the Gluster community when he started experimenting with the integration between oVirt (http://ovirt.org/) and GlusterFS. Long a proponent of converged infrastructure, Theron bring years of expertise from his stints at VMware and Nexenta.

    Supporting Quotes

    John Mark Walker, Gluster Community Leader, Red Hat

    “The additions of Citrix and Harvard FASRC to the Gluster Community show that we continue to build momentum in the software-defined storage space. With the continuing integration with all cloud and big data technologies, including the Xen Hypervisor and CloudStack, we are building the default platform for modern data workloads.

    Mark Hinkle, Senior Director, Open Source Solutions, Citrix

    “We see an ever increasing hunger for storage solutions that have design points that mirror those in our open source and enterprise cloud computing efforts. Our goal is to enable many kinds of storage with varying levels of utility and we see GlusterFS as helping to pioneer new advances in this area. As an active participant in the open source community we want to make sure projects that we sponsor like Apache CloudStack and the Linux Foundation’s Xen Project are enabled to collaborate with such technologies to best serve our users.

    James Cuff, Assistant Dean for Research Computing, Harvard University

    “As long term advocates of both open source, and open science initiatives at scale, Research Computing are particularly excited to participate on the Gluster Community Governing Board. We really look forward to further accelerating science and discovery through this important and vibrant community collaboration.”

     

    ***The OpenStack mark is either a registered trademark/service mark or trademark/service mark of the OpenStack Foundation, in the United States and other countries, and is used with the OpenStack Foundation’s permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community

    ***Gluster and GlusterFS are trademarks of Red Hat, Inc.

    ***Xen and Linux are trademarks of The Linux Foundation

    ***Apache Cloudstack is a trademark of the Apache Software Foundation

  • Gluster Cloud Night Amsterdam

    Join us on March 4 for the Gluster Community seminar and learn how to improve your storage.
    This half day seminar brings you in-depth presentations, use cases, demos and developer content presented by Gluster Community experts.

    REGISTRATION

    Register today for this free half-day seminar and reserve your seat since spaces are limited. Click here to register.

    We look forward to meeting you on March 4th!

    AGENDA

    13:30 – 13:45 Registration
    13:45 – 14:15 The State of the Gluster Community
    14:15 – 15:30 GlusterFS for SysAdmins, Niels de Vos, Red Hat
    15:30 – 15:45 Break
    15:45 – 16:30 Adventures in Cloud Storage with OpenStack and GlusterFS
    Tycho Klitsee (Technical Consultant and Co-owner) of Kratz Business Solutions
    16:30 – 17:15 Gluster Forge Demos, Fred van Zwieten, Technical Engineer, VX Company and Marcel Hergaarden, Red Hat
    17:15 – 18:00 Networking Drinks

  • Gluster Spotlight on James Shubin: Puppet-Gluster, Vagrant and GlusterFS Automation

    [youtube http://www.youtube.com/watch?v=SV1_RpZssGk&w=560&h=315]

    ***UPDATE: Due to weather-related flight cancelations and rebooking, we had to push this back to Thursday, January 23, at noon PST/3pm EST/20:00 GMT***

    James Shubin is known in the Gluster community for his work on the Puppet-Gluster module.

    Recently, he’s begun to create powerful cocktails of Puppet and Vagrant to create recipes for automated Gluster deployments. See, eg.

    Building Base Images for Vagrant with a Makefile

    and

    Testing GlusterFS During GlusterFest

    This will be quite a fun spotlight, and very much worth your while. As usual, join the #gluster-meeting channel on the Freenode IRC network to participate in the live Q&A.

    About Gluster Spotlight

    Gluster Spotlight is a weekly Q&A show featuring the most exciting movers and shakers in the Gluster Community. If you don’t catch them live, you can always watch the recordings later.

  • GlusterFest Weekend is Here – Jan 17 – 20

    As I mentioned yesterday, the GlusterFest is nigh. This time, we’ll break out testing into two types:
    • Performance testing
    • Feature testing
    To learn about the GlusterFest and what it is, visit the GlusterFest home at gluster.org/gfest
    Remember that if you file a bug that is verified by the Gluster QE team, you’ll win a t-shirt plus other swag.

    PERFORMANCE TESTING

    We are lucky in that two individuals have stepped up with tools to help with performance testing. One is James Shubin with his Puppet-Gluster module:
    https://forge.gluster.org/puppet-gluster/

    Together with his blog posts on puppet-gluster + vagrant, you should have an easy way to deploy GlusterFS:
    Automatically deploying GlusterFS with Vagrant and Puppet

    Also, Ben England recently released some code for his Smallfile performance testing project, which targets metadata-intensive workloads:

    http://forge.gluster.org/smallfile-performance-testing

    He also wrote up a nice primer on performance testing on the Gluster.org wiki that discusses iozone, smallfile, and how to utilize performance testing in general:
    http://www.gluster.org/community/documentation/index.php/Performance_Testing
    Please follow the instructions on the GlusterFest page (gluster.org/gfest) and report your results there. Some of the test results are quite large, so you will want to report test results on a separate page, either on the Gluster.org wiki or on the paste site of your choosing, such as fpaste.org.
    Please file any bugs and report them on the gluster-devel list, as well as providing links on the GlusterFest page.

    FEATURE TESTING

    In addition to performance, we have new features in 3.5 which needs some further testing. Please follow the instructions on the GlusterFest page and add your results there. Some of the developers were kind enough to include testing scenarios with their feature pages. If you want your feature to be tested but didn’t supply any testing information, please add that now.

    The GlusterFest begins at 00:00 GMT/UTC (today, January 17) and ends at 23:59 GMT/UTC on Monday, January 20.
    Rev your engines and get ready for some testing!

  • GlusterFS 3.5 Beta + GlusterFest Weekend

    The first GlusterFS 3.5 Beta is here! See what features made it in over at the 3.5 planning page. Here are some of the marquee features:

    With this first beta, we’ll have the next weekend GlusterFest! We’ll kick it off on Friday, January 17 at 00:00 GMT, continuing through Monday, January 20 at 23:59 GMT. Set your clocks!

  • Gluster Hangout with Daniel Mons from Cutting Edge

    [youtube http://www.youtube.com/watch?v=Ep4C2XWsG8o&w=560&h=315]

    Dan Mons came across GlusterFS at his job with Cutting Edge, a VFX company. He needed lots of storage space that was available to many different users – and he needed it to be able to expand as he needed. That it was free and ran on commodity systems was a big plus.

    Come join us as we learn from Dan and pepper him with lots of questions. We’ll be at a special time this week because Dan is in Oz – 5pm Pacific US/8pm Eastern US/01:00 GMT

    Follow along on YouTube in the video above and ask questions in #gluster-meeting on the Freenode IRC network (irc.freenode.net or irc.gnu.org, among others).

  • Hangout with Semiosis (Louis Z) Today – Gluster on AWS, Java Filesystem and more

    In about 90 minutes, Louis Zuckerman and I will be “hanging out” and talking about how he came to deploy GlusterFS on AWS, and why he’d developing a Java Filesystem integration with GlusterFS. I’ll post the embedded YouTube link here when we’re about to go live. Hangout starts at 11am EST, 8am PST, 16:00GMT – follow along on YouTube and ask questions in #gluster-meeting on IRC.gnu.org.

     

    [youtube http://www.youtube.com/watch?v=usoY_FPc2EY&w=560&h=315]