Planet JM

Author: johnmark

  • Moving on From Gluster

    All good things must come to an end. I can say with no equivocation that the last three years have been the most rewarding from a work perspective than any other job I’ve ever had. When I accepted this challenge in May, 2011, I had no idea that the project and community would blossom as they have. I had no idea how many great people were already in place to push this project to the forefront of open source development. I had no idea how many great partners we would find who share our vision for open source storage. I also, of course, didn’t know that Gluster, Inc. would be acquired within months of my arrival, which drastically increased the velocity of the project and community. I didn’t know any of that – what I did know was that there was a pretty cool project called GlusterFS and it seemed like the way forward for storage.

    After we were acquired, we knew there would be a bit of angst from the community about whether we would still actively support other distributions outside of the Red Hat arena. I’m proud to say that we have done that, with active contributions from various community members for Ubuntu, Debian, NetBSD and OpenSUSE builds. We always strove to make gluster.org a truly open community and, in some respects, “bigger than Red Hat.”

    Along the way, we created a board consisting of active community members and organizations. We made the project more transparent and active than ever. We greatly increased the degree that the community is a collaborative experience beyond just the immediate development team. And we greatly increased the reach and scope of the open source storage ecosystem. I can look back and feel privileged to have worked with such amazing visionaries, developers and community evangelists.

    Now it’s time to turn the Gluster community over to someone who can build on what we’ve done and take it even further. I’m staying at Red Hat but moving on to other projects and communities. The ideal candidate should know their way around open source projects and communities, should have an unyielding desire to push things beyond the status quo, should know a thing or two about business strategy, and should understand how to identify which organizations should invest in a community and sell them on the vision. As I’ve mentioned before, today’s community leaders are the equivalent of startup executives, having to mesh together product management and marketing, business development and strategy, sales and messaging into a cohesive whole.

    Are you the next Gluster Community Leader? Drop me a line on IRC – I’m “johnmark” on the Freenode network.

  • The Rise of Open Source Analytics Software

    I was pleased to read about the progress of Graylog2, ElasticSearch, Kibana, et al. in the past year. Machine data analysis has been a growing area of interest for some time now, as traditional monitoring and systems management tools aren’t capable of keeping up with All of the Things that make up many modern workloads. And then there are the more general purpose, “big data” platforms like Hadoop along with the new in-memory upstarts sprouting up around the BDAS stack. Right now is a great time to be a data analytics person, because there has never in the history of computing been a richer set of open source tools to work with.

    There’s a functional difference between what I call data processing platforms, such as Hadoop and BDAS, and data search presentation layers, such as what you find with the ELK stack (ElasticSearch, Logstash and Kibana). While Hadoop, BDAS, et al. are great for processing extremely large data sets, they’re mostly useful as platforms for people Who Know What They’re Doing (TM), ie. math and science PhDs and analytics groups within larger companies. But really, the search and presentation layers are, to me, where the interesting work is taking place these days: it’s where Joe and Jane programmer and operations person are going to make their mark on their organization. And many of the modern tools for data presentation can take data sets from a number of sources: log data, JSON, various forms of XML, event data piped directly over sockets or some other forwarding mechanism. This is why there’s a burgeoning market around tools that integrate with Hadoop and other platforms.

    There’s one aspect of data search presentation layers that has largely gone unmentioned. Everyone tends to focus on the software, and if it’s open source, that gets a strong mention. No one, however, seems to focus on the parts that are most important: data formats, data availability and data reuse. The best part about open source analytics tools is that, by definition, the data outputs must also be openly defined and available for consumption by other tools and platforms. This is in stark contrast to traditional systems management tools and even some modern ones. The most exciting premise of open source tooling in this area is the freedom from the dreaded data roach motel model, where data goes in, but it doesn’t come out unless you pay for the privilege of accessing the data you already own. Recently, I’ve taken to calling it the “skunky data model” and imploring people to “de-skunk their data.”

    Last year, the Red Hat Storage folks came up with the tag line of “Liberate Your Information.” Yes, I know, it sounds hokey and like marketing double-speak, but the concept is very real. There are, today, many users, developers and customers trapped in the data roach motel and cannot get out, because they made the (poor) decision to go with a vendor that didn’t have their needs in mind. It would seem that the best way to prevent this outcome is to go with an open source solution, because again, by definition, it is impossible to create an open source solution that creates proprietary data – because the source is open to the world, it would be impossible to hide how the data is indexed, managed, and stored.

    In the past, one of the problems is that there simply weren’t a whole lot of choices for would-be customers. Luckily, we now have a wealth of options to choose from. As always, I recommend that those looking for solutions in this area go with a vendor that has their interests at heart. Go with a vendor that will allow you to access your data on your terms. Go with a vendor that gives you the means to fire them if they’re not a good partner for you. I think it’s no exaggeration to say that the only way to guarantee this freedom is to go with an open source solution.

    Further reading:

     

  • Citrix and Harvard FASRC Join Gluster Community; Board Expands

    Citrix, Harvard University FASRC and long-time contributors join the Gluster Community Board to drive the direction of open software-defined storage

    February 5, 2014 – The Gluster Community, the leading community for open software-defined storage, announced today two new organizations have signed letters of intent to join: Citrix, Inc. and Harvard University’s Faculty of Arts and Science Research Computing (FASRC) group. This marks the third major expansion of the Gluster Community in governance and projects since mid-2013. Monthly downloads of GlusterFS have tripled since the beginning of 2013, and traffic to gluster.org has increased by over 50% over the previous year. There are now 45 projects on the Gluster Forge and more than 200 developers, with integrations either completed or in the works for Swift, CloudStack, OpenStack Cinder, Ganeti, Archipelago, Xen, QEMU/KVM, Ganesha, the Java platform, and SAMBA, with more to come in 2014.

    Citrix and FASRC will be represented by Mark Hinkle, Senior Director of Open Source Solutions, and James Cuff, Assistant Dean for Research Computing, respectively, joining two individual contributors: Anond Avati, Lead GlusterFS Architect, and Theron Conrey, a contributing speaker, blogger and leading advocate for converged infrastructure. Rounding out the Gluster Community Board are Xavier Hernandez (DataLab); Marc Holmes (Hortonworks), Vin Sharma (Intel), Jim Zemlin (The Linux Foundation), Keisuke Takahashi (NTTPC), Lance Albertson (The Open Source Lab at Oregon State University), John Mark Walker (Red Hat), Louis Zuckerman, Joe Julian, and David Nalley.

    Citrix

    Citrix has become a major innovator in the cloud and virtualization markets. They will drive ongoing efforts to integrate GlusterFS with CloudStack (https://forge.gluster.org/cloudstack-gluster) and the Xen hypervisor. Citrix is also sponsoring Gluster Community events, including a Gluster Cloud Night at their facility in Santa Clara, California on March 18.

    Harvard FASRC

    The research computing group at Harvard has one of the largest known deployments of GlusterFS in the world, pushing GlusterFS beyond previously established limits. Their involvement in testing and development has been invaluable for advancing the usability and stability of GlusterFS.

    Anand Avati

    Anand Avati was employee number 3 at Gluster, Inc. in 2007 and has been the most prolific contributor to the GlusterFS code base as well as its most significant architect over the years. He is primarily responsible for setting the roadmap for the GlusterFS project. Avati is employed by Red Hat but is an individual contributor for the board.

    Theron Conrey

    Theron became involved in the Gluster community when he started experimenting with the integration between oVirt (http://ovirt.org/) and GlusterFS. Long a proponent of converged infrastructure, Theron bring years of expertise from his stints at VMware and Nexenta.

    Supporting Quotes

    John Mark Walker, Gluster Community Leader, Red Hat

    “The additions of Citrix and Harvard FASRC to the Gluster Community show that we continue to build momentum in the software-defined storage space. With the continuing integration with all cloud and big data technologies, including the Xen Hypervisor and CloudStack, we are building the default platform for modern data workloads.

    Mark Hinkle, Senior Director, Open Source Solutions, Citrix

    “We see an ever increasing hunger for storage solutions that have design points that mirror those in our open source and enterprise cloud computing efforts. Our goal is to enable many kinds of storage with varying levels of utility and we see GlusterFS as helping to pioneer new advances in this area. As an active participant in the open source community we want to make sure projects that we sponsor like Apache CloudStack and the Linux Foundation’s Xen Project are enabled to collaborate with such technologies to best serve our users.

    James Cuff, Assistant Dean for Research Computing, Harvard University

    “As long term advocates of both open source, and open science initiatives at scale, Research Computing are particularly excited to participate on the Gluster Community Governing Board. We really look forward to further accelerating science and discovery through this important and vibrant community collaboration.”

     

    ***The OpenStack mark is either a registered trademark/service mark or trademark/service mark of the OpenStack Foundation, in the United States and other countries, and is used with the OpenStack Foundation’s permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community

    ***Gluster and GlusterFS are trademarks of Red Hat, Inc.

    ***Xen and Linux are trademarks of The Linux Foundation

    ***Apache Cloudstack is a trademark of the Apache Software Foundation

  • Gluster Cloud Night Amsterdam

    Join us on March 4 for the Gluster Community seminar and learn how to improve your storage.
    This half day seminar brings you in-depth presentations, use cases, demos and developer content presented by Gluster Community experts.

    REGISTRATION

    Register today for this free half-day seminar and reserve your seat since spaces are limited. Click here to register.

    We look forward to meeting you on March 4th!

    AGENDA

    13:30 – 13:45 Registration
    13:45 – 14:15 The State of the Gluster Community
    14:15 – 15:30 GlusterFS for SysAdmins, Niels de Vos, Red Hat
    15:30 – 15:45 Break
    15:45 – 16:30 Adventures in Cloud Storage with OpenStack and GlusterFS
    Tycho Klitsee (Technical Consultant and Co-owner) of Kratz Business Solutions
    16:30 – 17:15 Gluster Forge Demos, Fred van Zwieten, Technical Engineer, VX Company and Marcel Hergaarden, Red Hat
    17:15 – 18:00 Networking Drinks

  • Gluster Spotlight on James Shubin: Puppet-Gluster, Vagrant and GlusterFS Automation

    [youtube http://www.youtube.com/watch?v=SV1_RpZssGk&w=560&h=315]

    ***UPDATE: Due to weather-related flight cancelations and rebooking, we had to push this back to Thursday, January 23, at noon PST/3pm EST/20:00 GMT***

    James Shubin is known in the Gluster community for his work on the Puppet-Gluster module.

    Recently, he’s begun to create powerful cocktails of Puppet and Vagrant to create recipes for automated Gluster deployments. See, eg.

    Building Base Images for Vagrant with a Makefile

    and

    Testing GlusterFS During GlusterFest

    This will be quite a fun spotlight, and very much worth your while. As usual, join the #gluster-meeting channel on the Freenode IRC network to participate in the live Q&A.

    About Gluster Spotlight

    Gluster Spotlight is a weekly Q&A show featuring the most exciting movers and shakers in the Gluster Community. If you don’t catch them live, you can always watch the recordings later.

  • GlusterFest Weekend is Here – Jan 17 – 20

    As I mentioned yesterday, the GlusterFest is nigh. This time, we’ll break out testing into two types:
    • Performance testing
    • Feature testing
    To learn about the GlusterFest and what it is, visit the GlusterFest home at gluster.org/gfest
    Remember that if you file a bug that is verified by the Gluster QE team, you’ll win a t-shirt plus other swag.

    PERFORMANCE TESTING

    We are lucky in that two individuals have stepped up with tools to help with performance testing. One is James Shubin with his Puppet-Gluster module:
    https://forge.gluster.org/puppet-gluster/

    Together with his blog posts on puppet-gluster + vagrant, you should have an easy way to deploy GlusterFS:
    Automatically deploying GlusterFS with Vagrant and Puppet

    Also, Ben England recently released some code for his Smallfile performance testing project, which targets metadata-intensive workloads:

    http://forge.gluster.org/smallfile-performance-testing

    He also wrote up a nice primer on performance testing on the Gluster.org wiki that discusses iozone, smallfile, and how to utilize performance testing in general:
    http://www.gluster.org/community/documentation/index.php/Performance_Testing
    Please follow the instructions on the GlusterFest page (gluster.org/gfest) and report your results there. Some of the test results are quite large, so you will want to report test results on a separate page, either on the Gluster.org wiki or on the paste site of your choosing, such as fpaste.org.
    Please file any bugs and report them on the gluster-devel list, as well as providing links on the GlusterFest page.

    FEATURE TESTING

    In addition to performance, we have new features in 3.5 which needs some further testing. Please follow the instructions on the GlusterFest page and add your results there. Some of the developers were kind enough to include testing scenarios with their feature pages. If you want your feature to be tested but didn’t supply any testing information, please add that now.

    The GlusterFest begins at 00:00 GMT/UTC (today, January 17) and ends at 23:59 GMT/UTC on Monday, January 20.
    Rev your engines and get ready for some testing!

  • GlusterFS 3.5 Beta + GlusterFest Weekend

    The first GlusterFS 3.5 Beta is here! See what features made it in over at the 3.5 planning page. Here are some of the marquee features:

    With this first beta, we’ll have the next weekend GlusterFest! We’ll kick it off on Friday, January 17 at 00:00 GMT, continuing through Monday, January 20 at 23:59 GMT. Set your clocks!

  • Gluster Hangout with Daniel Mons from Cutting Edge

    [youtube http://www.youtube.com/watch?v=Ep4C2XWsG8o&w=560&h=315]

    Dan Mons came across GlusterFS at his job with Cutting Edge, a VFX company. He needed lots of storage space that was available to many different users – and he needed it to be able to expand as he needed. That it was free and ran on commodity systems was a big plus.

    Come join us as we learn from Dan and pepper him with lots of questions. We’ll be at a special time this week because Dan is in Oz – 5pm Pacific US/8pm Eastern US/01:00 GMT

    Follow along on YouTube in the video above and ask questions in #gluster-meeting on the Freenode IRC network (irc.freenode.net or irc.gnu.org, among others).

  • Hangout with Semiosis (Louis Z) Today – Gluster on AWS, Java Filesystem and more

    In about 90 minutes, Louis Zuckerman and I will be “hanging out” and talking about how he came to deploy GlusterFS on AWS, and why he’d developing a Java Filesystem integration with GlusterFS. I’ll post the embedded YouTube link here when we’re about to go live. Hangout starts at 11am EST, 8am PST, 16:00GMT – follow along on YouTube and ask questions in #gluster-meeting on IRC.gnu.org.

     

    [youtube http://www.youtube.com/watch?v=usoY_FPc2EY&w=560&h=315]

  • On Millennials and Special Snowflakes

    I’m really sick and tired of all the boomer and gen X tsk tsk-ing those dreaded me-first millennials. All of these articles, which you’ve no doubt seen by now, follow a similar pattern:

    • Start with a reference to “too much praise” and too much emphasis on self-esteem. Bonus points if the writer mentions “everyone gets a trophy”
    • Then the writer lays down the hammer! And writes as if they’ve stumbled upon some grand discovery! “You’re no special snowflake” indeed.
    • Cue up some old-school expressions of insecurity that the writer mistakes for “tough love.” Back in my day, grown-ups rubbed our faces in our own failure. And we loved it! It was character building!
    • Add in some almost too revealing subtext of bitterness upon entering middle age while accomplishing zilch in the writer’s lifetime. Which, of course, has NOTHING AT ALL to do with the writer’s irrational jealousy of a whole generation of kids. NOTHING AT ALL, I SAY.
    • Add a sprinkling of cherry-picked facts to support the argument. Because I said so, that’s why. ZOMG, millenials will sic their parents on you when they fail!
    • Get off my lawn!

    The most heinous example of this was David McCullough Jr., an English teacher at Wellesley High School in Massachusetts, who stupidly masturbated to his own self-importance at a high school commencement address, wagging his finger and telling them all about the BIG SCARY REAL WORLD and how “they’re not special!” Contrary to what many commentators may have written, this is not some refreshing new phenomenon, and I remember it well – adults who couldn’t get over their own failures trying to cut us down to size to help them cope with their own failures, insecurities and underachievement. We’re afraid that the next generation may show us up, so we better chop them down while there’s still time.

    There’s a lot to be said about how middle class values have evolved over the decades, evolving from working class, blue collar families to white collar workers looking to get ahead. At every single stage, with every hand-off from the preceding generation, the new kids were always told to a.) get an education so they could improve upon the older generation b.) follow their dreams, unlike their elders, who didn’t have that luxury and c.) marry for love, not settle for whoever happens to be around. In the boomer generation, that means improving upon your parents farming or blue collar background. In the “gen x” days, it meant bettering your parents service jobs or blue collar history.

    And now it means… what, exactly? Seriously, if the whole “improve on your parents outcome” has been completely baked into every facet of society, what did we expect out of this current crop of kids? Pretty much everyone with some means has gone to college and worked in the white collar world their entire post-education lives. And now, with a present and future of very limited growth, the world is an extremely competitive place, rife with fear, loathing and self-doubt, which manifests itself in a variety of ways. Parents are hell-bent on making sure their kids “get ahead”, so they enroll their kids in every competitive sport, spend exorbitant amounts of money to send their kids to the best public schools, enroll them in art and music programs, and push them into every extra-curricular thing they can afford. The archetype of the millenials’ parents is not the coddling, always-praising sunshine pumper, it’s the tiger mom. And the implicit message to the kids is, “you had better succeed. You cannot fail – you’re our only hope to maintain our status. And we will do *anything* to protect that hard-fought status.”

    I call outright BS on the special snowflake business. That’s not the problem. They’ve been taught since they were plucked from a crib to go to art class that this is a hyper-competitive world in ways that it never was before. I do not understand how anyone can look at the lives of young adults who came of age in the 90’s and 00’s and come away with the impression that they didn’t face enough consequences of losing. They faced those consequences quite often, thank you very much – from the first time they learned they weren’t in the G&T classes, to the first time they didn’t make a sports team, to the times their teams, in whatever activity, didn’t win. There is now an unprecedented pressure on the middle class, not to mention baked-in anxiety of of falling down the status pole, with many people from below trying to rise up to the “American Dream.” Then add to that the stories shared by parents about that other kid/cousin/neighbor down the street who’s accomplished some amazing thing, evoking pangs of anxiety and jealousy from their kids. To magnify that effect, there’s now social media with its pervasive humble braggers to drive home the point that you’re a loser, baby, and no special snowflake. The pressure on millennials is about performing up to higher standards, achieving perfection, and making the impossible possible.

    There’s an interesting psychological concept that manifests itself in particularly pernicious ways as society becomes more taxed by inequality. People tend to focus on those who are above them in status, which means that they don’t even notice those who are lower in status. The result is that the more successful you are, the more elusive success becomes, due to the goalposts of success always being in motion. You don’t notice those peers you’ve just joined – only those who are at a level above you. So, parents who may have started from modest roots never quite appreciate the distance they’ve traveled, and this class anxiety transfers seamlessly to their progeny. And that anxiety, and how they deal with it, explains many of the complaints you hear about millennials.

    I don’t think millennials think they’re special snowflakes. I think they’re scared shitless that they’re losing. The era of lowered expectations means that everything they’ve been told when they were growing up is a complete lie. There is no attainable success – it’s always elusive, just beyond your grasp. Having established that, forgive them if, frankly, they don’t feel the need to jump through your horseshit hoops because, ultimately, it doesn’t really amount to anything substantial. Forgive them if they’d rather pursue their wild-ass dreams instead of whatever you think they *should* be doing. Whenever I read articles full of free, unsolicited “advice” for millennials, I’m reminded of my med-school friends who told me about the horrors of 36-hour shifts. When I asked them why they continued this practice despite its obvious potential for failure, their response was simple: because their predecessors had to go through it, it was only “fair” that this next crop go through the same self-abuse. Lather, rinse, repeat. Got that? It’s not about better results, it’s about equal suffering. So no, I don’t think millennials are a blight on humanity. In fact, I think they’re our best hope for a sane, future work-life balance.

    Because they, more than most, understand that our current systems of blind obeisance to fascistic organizations should not be a mandatory rite-of-passage that everyone follow. Rather, we should seriously analyze how we got where we are, study the results of our current systems, and if need be, either reform them or start over. Don’t fight the millennials – pay attention to what they have to say.