Talks and Papers

Most of my conference and user group talks can be found at my GitHub Presentations repo.

Stream All the Things!!

Software Architecture Conference NYC, April 2017, Strata London, May 2017, and Strata NYC, September 2017

I discuss the emerging architecture for large-scale stream data processing, that also integrates the best of microservice architectures.

Download PDF Watch video (Coming Soon)

The SMACK Stack: Emerging Fast Data and Microservice Architectures

CodeNode London, October 24, 2016

Spark, Mesos, Akka, Cassandra, and Kafka form the basis for new, flexible architectures for Fast Data and Microservice applications. This talk describes the characteristics these applications need and how the SMACK combination meets those needs.

Scala and the JVM for Big Data: Lessons from Spark

Strata + Hadoop World, San Jose and Singapore, 2016

The JVM is the standard platform for Big Data and Scala is emerging as the standard programming language for Big Data Developers, driven in part by Spark. What lessons can we draw from this picture?

Spark 2.0

Chicago Spark Meetup, May 18, 2016

Spark 2.0 introduces significant improvements and refactorings. This talk summarizes the major changes.

Why Spark Is The Next Top (Compute) Model

Numerous Venues - 2014, 2015

Spark has emerged as the replacement for MapReduce in Hadoop applications. This talk explains why.

Spark on Mesos

Strata + Hadoop World London and NYC 2015

While Spark is now popular on Hadoop, managed by YARN, it emerged as demonstration project for Mesos. This talk explores Mesos, compares it to YARN, and argues for why you should consider a Spark + Mesos cluster.

Download PDF Watch video (Coming Soon)

Data Science at Scale with Spark

GOTO Chicago 2015

Using examples, I show how to use Spark for Data Science at scale in ways that were previously not feasible with other tools.

Download PDF Watch video (Coming Soon)

The Unreasonable Effectiveness of Scala for Big Data

Scala Days 2015.

Why Scala has proven so effective as the general-purpose programming language for Big Data development.

Error Handling in Reactive Systems

React Conf San Francisco 2014, Software Architecture 2015

Failure handling must be first-class in reactive systems, to satisfy the resilient trait. This talk discusses how reactive models and libraries support failure handling. Hint: Most only offer limited facilities. (March 19, 2015)

Download PDF Watch video (Coming Soon)

Reactive Design, Languages, and Paradigms

React Londong 2014 and YOW! LambdaJam 2014.

The Reactive Manifesto has four central traits. How well are they supported by Functional Programming, and Object-Oriented Programming, and Domain Driven Design? What about different languages? What about "reactive" approaches like Functional Reactive Programming, Reactive Extensions, and Actors? (April 8, 2014)

Copious Data, the Killer App for Functional Programming

LambdaJam Chicago

I argue that "Copious" Data (okay, Big Data) is driving adoption of Functional Programming (FP), more so than multicore concurrency concerns, because more developers will grapple with data problems than concurrency. Because FP is based on Mathematics, it is a natural fit for working with Data, whereas languages like Java, in which Hadoop is written, are poor choices. (November 21, 2013)

Download PDF Watch video (earlier version)

Reactive IoT

Thing Monk London 2013

This short talk describes why large-scale deployments of Internet of Things networks need to be managed with Reactive systems. (December, 2013)

SQL Strikes Back! Recent Trends in Data Persistence and Analysis

CodeMesh London 2014

Relational databases fell out of fashion with the rise of NoSQL and Hadoop. But SQL proved too useful for too many people, so there are now many SQL-based query tools for Hadoop and subsets of SQL on several "NoSQL" databases. This talk discusses this trend and why it started. (November 4th, 2014)

Download PDF Watch video (Coming Soon)

The Seductions of Scala

Various Venues

An introduction to Scala that I often give at conferences and user groups. The PDF includes a lot of extra material that won't fit into a 50-60 minute time slot. The GitHub page for this talk also has the sources used for the examples. In particular, for the Akka-based Actor example at the end of the talk, see README.md. (November 19, 2013)

MapReduce and Its Discontents

QCon NYC 2012, and Big Data Techcon Boston 2013

My first public talk where I claimed that MapReduce is the Enterprise JavaBeans of our time. I criticized the MapReduce programming model and the technical limitations of the Hadoop implementation, in particular. In part, I argued that Java (pre Java-8 especially) is the wrong tool for developing Big Data applications and middleware. Instead, we should be using Functional Programming, since when we work with data, we are really doing Mathematics! (April 11, 2013)

Why Big Data Needs to Be Functional

NE Scala Symposium 2011

A more general version of the previous "Discontents" talk, where I argue that the Hadoop community needs to drop reliance on Java-centric, Object-Oriented approaches and embrace Functional Programming and languages like Scala. (April 15th, 2012)

Heresies and Dogmas in Software Development

Strange Loop 2011

I look at 5 ideas in the history of software development that were once popular, and still are in some quarters, but are now seen by most people as obsolete. (November 9th, 2011)

Better Programming Through Functional Programming

A half-day tutorial that introduces Functional Programming, why it has become important for our time, and how you can apply its ideas in almost any language. Examples are given in Java and Ruby. There is also a shorter talk version. (July 31st, 2011)

Polyglot and Poly-paradigm Programming

QCon San Francicso 2008

An argument that modern development problems benefit from a multi-paradigm and/or multi-language solution strategy. Different strategies are discussed in the contexts of example problems (April 2, 2011)

Download PDF Watch video (Early version of this talk)

Hive - SQL for Hadoop

Chicago Hadoop Users Group

This talk introduces Hive, the original SQL tool for Hadoop and explains why it's a key technology that drove adoption of the ecosystem, primarily because it makes it easier to transition SQL-based data warehouses to Hadoop and it enables conventional data analysts to work with Hadoop. (January 2012)

H2O: An open-source, in-memory prediction engine for data science

Big Data Techcon San Francisco 2014

H2O is a high-performance, clustered compute engine for data science that fills an underserved niche in the Big Data world, working with data sets under a few TBs in size, which is just too big or too slow for a single machine to process, but not big enough to require Hadoop. This talk discusses why H2O is such a good fit for this niche. (April 1, 2014)

All My Presentations and Papers

Scala

The Seductions of Scala Adobe PDF.

I've given this presentation now at several venues. It provides a short introduction to some of the seductive features of Scala, like Traits for Mixin Composition, succinct syntax, and support for Functional Programming.

The Akka Framework

An overview of the Akka Framework for building robust, highly concurrent servers in Java or Scala. Note: This is a web-based presentation written using S5 (with hacks). See the instructions on the GitHub page.

Big Data

Why Spark Is The Next Top (Compute) Model Adobe PDF.

This is a talk for Emerging Technology for the Enterprise 2014. Spark has emerged as the most likely replacement for MapReduce in Hadoop applications. This talk explains why. (Last updated: April 23, 2014)

Spark on Mesos Adobe PDF

In this Strata + Hadoop World London and NYC 2015 talk I argue that, while Spark is now popular on Hadoop, managed by YARN, it emerged as demonstration project for Mesos. I explore Mesos, compare it to YARN, and argue for why you should consider a Spark + Mesos cluster. (Last updated: October 10, 2015

Scala and the JVM for Big Data: Lessons from Spark Adobe PDF

At the first Scala World 2015, I discussed how the JVM is the standard platform for Big Data and Scala is emerging as the standard programming language for Big Data Developers, driven in part by Spark. What lessons can we draw from this picture? (Last updated: October 10, 2015

Apache Spark and the Typesafe Reactive Platform: A Match Made in Heaven

In this lightning talk at the Spark Summit San Francisco 2014, I discuss why Spark and the Typesafe Reactive Platform are so complementary. (Last updated: July 1, 2014)

Copious Data: the Killer App for Functional Programming Adobe PDF Video.

I argue that "Copious" Data (okay, Big Data) will drive adoption of functional programming (FP), more than multicore concurrency concerns, because more developers will grapple with data problems than concurrency. Because FP is based on Mathematics, it is a natural fit for working with Data, whereas languages like Java, in which Hadoop is written, are poor choices. I gave the original version of this talk at LambdaJam Chicago, July 8, 2013. The video of that talk is on InfoQ. (Last updated: November 21, 2013)

SQL Strikes Back! Recent Trends in Data Persistence and Analysis Adobe PDF

Relational databases fell out of fashion with the rise of NoSQL and Hadoop. But SQL proved too useful for too many people, so there are now many SQL-based query tools for Hadoop and subsets of SQL on several "NoSQL" databases. This talk discusses this trend and why it started. CodeMesh London , November 4, 2014.

H2O: An open-source, in-memory prediction engine for data science Adobe PDF

H2O is a high-performance, clustered compute engine for data science that fills an underserved niche in the Big Data world, working with data sets under a few TBs in size, which is just too big or too slow for a single machine to process, but not big enough to require Hadoop. This talk discusses why H2O is such a good fit for this niche. Big Data Techcon San Francisco 2014.

MapReduce and Its Discontents Adobe PDF Video

I describe the MapReduce programming model and the technical limitations of the Hadoop implementation, in particular, in part because Java is the wrong tool for developing Big Data applications and middleware. Instead, we should be using Functional Programming, since we are really doing mathematics! I claim that Hadoop is the Enterprise JavaBeans of our time. The video of the talk is on InfoQ. (last update: December 2nd, 2012)

What's Ahead for Big Data? Adobe PDF

Big Data is at a crossroads, where Hadoop is showing its age and new compute models are emerging. NoSQL databases are adding query engines, sometimes based on SQL and NewSQL databases are bringing some measure of NoSQL performance to relational models. This talk examines these trends, focusing mostly on Hadoop today and tomorrow. I gave it at GOTO Chicago 2013 and later at CodeMesh 2013. (Last updated: December 4, 2013)

Why Big Data Needs to Be Functional Adobe PDF

I've been doing Hadoop-related consulting for the last year. Data analysis and processing is a perfect application of Functional Programming, yet a Java-centric, object-oriented mindset pervades that community. So, I did a 30-minute talk at the NE Scala Symposium on Why Big Data Needs to Be Functional. You can also find the video here.

Scalding for Hadoop Adobe PDF

The benefits of using Scalding, presented February 12 at the Chicago Hadoop User's Group (CHUG) after Paco Nathan presented on Cascading. Hence, my talk assumes some prior exposure to Cascading concepts. Even if you understand Cascading concepts, I recommend reading Paco's presentation first. (last update: February 13nd, 2013)

Hive - SQL for Hadoop Adobe PDF

This talk for January, 2012 meeting of the Chicago Hadoop User's Group introduces Hive and explains why it's a key technology in the Hadoop ecosystem, primarily because it makes it easier to transition SQL-based data warehouses to Hadoop.

Reactive Programming

Error Handling in Reactive Systems Adobe PDF.

Failure handling must be "first-class" in reactive systems, to satisfy the resilient trait. This talk discusses how reactive models and libraries support failure handling. (Last updated: November 19, 2014)

Reactive Design, Languages and Paradigms Adobe PDF Video.

This is a talk first given at React 2014 and refined for YOW! LambdaJam 2014. It explores the meaning of Reactive Programming, as described in the Reactive Manifesto, and how well it is supported by general design paradigms, like Functional Programming, Object-Oriented Programming, and Domain Driven Design, by various languages, and by particular design approaches, such as Functional Reactive Programming, Reactive Extensions, Actors, etc. (Last updated: May 8, 2014)

Reactive IoT Adobe PDF.

This Thing Monk London 2013 lightning talk describes why large-scale deployments of Internet of Things networks need to be managed with Reactive systems. (Last updated: December, 2013)

Functional Programming

Better Programming through Functional Programming Adobe PDF

.

A half-day tutorial that introduces Functional Programming, why it has become important for our time, and how you can apply its ideas in almost any language. Examples are given in Java and Ruby. (last update: July 31st, 2011)

How Functional Programming Changes Developer Practices Adobe PDF

.

A talk at Agile 2011 on why Functional Programming improves our code and makes us more agile. This talk is adapted from my tutorial discussed above. (last update: July 31st, 2011)

Ruby

(Son of) Better Ruby Through Functional Programming Adobe PDF and Better Ruby Through Functional Programming Adobe PDF.

An updated and the original presentation I've given on applying functional programming ideas to Ruby development and how they improve your code. I gave the original talk at RubyConf 2008 (video) and the "Son of" talk at Windy City Rails 2009 (video). Note: the code samples are availabe here.

Acceptance Testing Java Applications with Cucumber, RSpec, and JRuby Adobe PDF

A tutorial that Aslak Hellesøy and I did August 27, 2009 at Agile 2009.

Principles of Ruby Application Design Adobe PDF

A talk I gave at Dr. Dobb's Architecture and Design World 2008. It discusses object-oriented and functional design principles that promote smaller, higher quality Ruby code. I don't discuss the "obvious" stuff, like how to use Rails, gems, JRuby, etc. Rather this talk is based on the premise that a successful architecture must be based on a foundation of good OO design, with a healthy dose of ideas from Functional Programming, Aspect-Oriented Design, etc. Feedback is welcome!

Aquarium

Aquarium: Aspect-Oriented Programming for Ruby Adobe PDF

This talk introduces Aquarium, an AOP framework for ruby. It discusses how Aquarium works and why AOP is interesting for Ruby. For example, I show a simple example of refactoring Rails code to improve it's clarity when Aquarium is used vs. using metaprogramming (e.g., class_eval and alias_method). There are a few exercises that go with the talk: Aquarium_RubyAOP_exercises.zip

Aquarium: AOP for Ruby Adobe PDF

This short talk (30 minutes) presented Aquarium to the aspect-oriented programming research community at the Aspect-Oriented Software Development 2008 Conference, in Brussels. The talk was based on my Industry Track paper. Note that some of the Aquarium APIs have changed since this talk and paper were prepared.

Modern Approaches to Software Development

A few years after I started my evolution from thinking in objects to thinking in functions, it occurred to me that our ideas of what's good and bad in software development have often changed. Today's dogma is tomorrow's heresy, if you will. So, I gave a talk on this idea at Strange Loop 2011 and subsequently in other venues.

Heresies And Dogmas In Software Development Adobe PDF Strange Loop 2011 Video (Yea, I know I used Comic Sans in that version...)

Five ideas in the history of software development that were once (or still are) popular, but now are (usually) seen as obsolete (plus a "bonus" topic). The video of the Strange Loop talk is on InfoQ.

I've done a series of talks on the theme of Polyglot and Poly-Paradigm Programming, where I discuss how using multiple languages and/or "paradigms" can simplify the solutions to common problems we encounter today in software developer.

Polyglot and Poly-Paradigm Programming Adobe PDF

Clean Systems: Clean Code at the Architecture Level Adobe PDF

How do you keep systems and architectures "clean"? This talk is based on the "Clean Systems" chapter I wrote for Bob Martin's new Clean Code book.

Don't Do This! How Not to Write Java Software Adobe PDF.

In this JavaOne 2009 talk, I discuss 10 mistakes I see Java teams make, why they are bad, and what to do instead.

General AOSD

AOP in Academia and Industry Adobe PDF

I discuss some of the academic research and industry trends that led to the creation of AOP, the current state of things, and how both can contribute to the future development of AOP and emerging technologies that might use it.

Aspect-Oriented Programming and Design for Java and AspectJ Adobe PDF

This is an introduction to AOP, given as a presentation at the Chicago Java User's Group, Sept. 6, 2007. I discuss why AOP is interesting, what problems it solves, followed by a look at how aspects can be implemented in AspectJ and Spring AOP. I then look at several object-oriented design principles adapted to aspects and, along the way, demonstrate fine-grained separation of concerns using AspectJ.

Aspect-Oriented Design Principles: Lessons from Object-Oriented Design (Adobe PDF on the AOSD 2007 Conference web site).

An examination of 11, well-established OOD principles and how they apply to AOD. The main objective of this paper is to address pragmatic issues of designing production-quality aspect-based software. In today's world, aspects won't prosper unless they promote agility, reusability, and maintainability. The 11 OOD principles discussed have proven to be effective design tools that promote these qualities (especially in languages like Java, C#, C/C++, etc.). I discuss how the principles apply to aspect software, how AOD refines them, and how they lead to new aspect-specific extensions that promote better aspect design. This paper is part of the Industry Track at AOSD 2007 in Vancouver, BC.

Contract4J

AOP@Work: Component Design with Contract4J
(IBM developerWorks).

An introduction to Contract4J and how to use it. The article also briefly discusses some emerging ideas about the nature of interfaces in Aspect-Oriented Design, and how Contract4J demonstrates some of those ideas.

The Challenges of Writing Reusable and Portable Aspects in AspectJ: Lessons from Contract4J Adobe PDF.

Writing Contract4J revealed many of the challenges faced when trying to write generic, reusable aspects in AspectJ. This paper discusses those challenges, how they were addressed, and offers suggestions for improvements to AspectJ and our concepts of AOD that will make it easier to write reusable aspects. Presented at the Industry Track, AOSD 2006, Bonn Germany, March 22, 2006 (Click here to get all the Industry Track papers).

Contract4J for Design by Contract in Java: Design Pattern-Like Protocols and Aspect Interfaces Adobe PDF

Contract4J comes in two forms; an implementation where contract tests are defined using Java 5 annotations and a more experimental implementation where the tests are defined as regular Java methods with a JavaBeans-like naming convention. This paper discusses how these two forms define a protocol with two different syntaxes that allow aspects to advice objects with minimal coupling, in the spirit of interface design. Presented at the ACP4IS Workshop at AOSD 2006, Bonn Germany, March 21, 2006.

Older Talks and Papers

The Future of AOP Adobe PDF

Predictions about the likely future of AOSD, based on the history of OOP. The basis for this comparison is the conceptual parallels between the two technologies. The presentation also has a brief description of AOP itself. It ends with an appraisal of the current state of AOP and offers some recommendations for how to begin working with it today.

Use Cases as Aspects Adobe PDF

Some preliminary work on Ivar Jacobson's idea of considering Use Case-oriented software development as a form of AOSD.

AOP and MDA Adobe PDF

An examination of how AOSD addresses some technical challenges in OMG's Model Driven-Architecture (MDA).

</section>