These are some of my short-term predictions for AI, and a bit of my reasoning behind each prediction.

Context Platforms are Coming

I believe LLMs like ChatGPT and Bard are the most sophisticated systems in existence, but I mean that in the classical definition of "sophistry" where these systems are incredibly good at producing arguments that sound clever and feasible but are often without reasoning, logic, or facts.

LLMs have the major challenge of "hallucinations" in the model outputs where they will provide answers to questions that seem well-reasoned but are factually or logically incorrect. Google's Bard model famously gave a factually inaccurate answer in one of their advertisements.

This isn't surprising, as OpenAI states explicitly regarding ChatGPT:

1. The model was not trained on data past December 2021, so it is relatively stale.

2. "ChatGPT may produce inaccurate information about people, places, or facts"

That said, it turns out that when you provide these LLMs with explicit context for what you want to answer along with your prompt then that seems to overcome these issues for a majority of cases. Stephen Wolfram of WolframAlpha and Mathematica wrote about how his platform could be utilized to provide factual context to correct GPT.

There are two ways that I consider context concerning LLMs:

1. What relevant information do I need to answer my query?

2. How can I phrase my query to get the response I want?

Relevant Information

We know that when we explicitly provide relevant information to our LLMs we get far more relevant responses. This is so simple I will demonstrate it here with the text from this article on SBF:

...

Now the question becomes "How do I get a relevant chunk of text?"

One way is that you could search and find an article or page that you think is relevant and feed that into ChatGPT as explicit context and then ask it your question.

This method works pretty well, and some very cool projects like ChatPDF are built on this idea of just explicitly asking for the context (in that case, a PDF document) that a person wants to interact with. Unfortunately, this doesn't scale across the thousands of documents and websites that are utilized within an enterprise.

Relevant Information... at Enterprise Scale!

"Well, let's just throw all of our documents in as context."

That works to an extent. Unfortunately, GPT has limits on the size of the prompt that prevent individuals from throwing more than just a few documents into a prompt at any given time. Not to mention cost scales with the size of the prompt, making this method increasingly expensive.

Fortunately, rapid information retrieval across massive document storage has been an active area of research and development for decades. Popularity with LLMs has resulted in renewed interest in developing and productizing these systems further for usage in concert.

Probably the most prevalent method right now is searching across text embeddings which seems to have the advantage of preserving forms of semantics (not just the words but their relationships and intentions) from a user query that is entered into the search. The details of how it works are rather academic but I think Cohere does a good job of explaining things. One important thing to understand is that this approach takes sentences (which are very hard to compare) and simplifies them into sets of numerical vectors that are much easier to compare.

So far OpenAI has services like its built-in embeddings API to help with this issue but I don't believe they are focusing efforts on this and are rather focused on growing the capabilities of their LLMs. New products and even pivots of existing products are emerging to generate, store, and search these vectors at enterprise scale like Weaviate and Pinecone.

Relevant Queries

Now that we know we can rapidly find and use information that can answer our prompts, another challenge is that the phrasing of your prompts can be even more important in some cases. Niklas Heidloff gives great examples of this out in the wild.

What has resulted is "prompt engineering": a skillset of crafting the right LLM prompts to get intended outcomes. A market has emerged for some of the most successful prompts. Even the CEO of OpenAI himself has recognized that this is a learnable skill set that is increasingly valuable.

As with all skills that rely on data and information, certain components of the skillset will become formalized and productized. Langchain is a growing toolset based around prompt engineering that allows developers to experiment and template with prompts, pass information between LLMs and other systems like Wolfram Alpha, and even "chain" prompts together to perform more complex tasks.

Tools that retrieve information and manage prompts can be combined either in frameworks or fully-managed platforms. These "context platforms" are in the early stages; right now we are seeing the deep technical implementation that individuals will build applications around, rather than first-class products with solutions vertically integrated, but I believe by 2024 we will see our first platform that marries these concepts together and sells a whole solution to customers.

LLMs will be Interchangeable

OpenAI has recognized and openly spoken about all the limitations I've pointed out above. While they continue to invest in some interfaces to their LLMs (such as the ChatGPT website) and some helper tools, I believe their core investment will be in improving and serving their models as APIs to broad consumers. This is both due to product-business alignment and also because they have a growing swatch of paying customers who use these APIs to build competing interfaces. Focusing on LLM performance will be increasingly important as the battle for AI supremacy heats up between some of the biggest players.

Where we will see some of the biggest startups, innovations, and creative applications will be in the clever combination of LLMs and the aforementioned Context Platforms to tackle problem sets stratified by industry. As such, I think these LLMs will become interchangeable either through development work on the part of the companies or platform owners to write the middleware necessary, or an agreed API standardization between companies.

Much like cloud providers, operations will be relatively interchangeable between LLM providers. Companies will be able to switch LLMs when one demonstrates higher performance or possibly better data privacy and compliance (a FEDRAMPed LLM, anyone?).

I welcome discussion here, as this is probably the most controversial of my takes.

LLMs Are Going Behind the Firewall

For those curious, my prompt in Playground AI for the above image was "CIA Agent Jack Ryan working with a Cyberpunk Artificial Intelligence Robot".

When GPT models were first being served by OpenAI, curious minds began deducing the potential costs of the computing required to train and run these models. Projections ran roughly into millions of dollars to do initial training, and hundreds of thousands of dollars to host and maintain each month. Pocket change for the likes of Google and Microsoft but unapproachable for your average startup.

Then something incredible happened: Facebook/Meta's own LLM Llama was leaked. Llama is significantly smaller, and with fine-tuning many are finding that it can approach ChatGPT's capabilities for specified tasks.

While there are still questions about licensing, it's fair to say that the cat is out of the proverbial bag and initial skepticism about LLMs running in edge and air-gapped environments is being sufficiently squashed with new increases and performance and optimization around self-hosted LLMs popping up every week (they can run on a Pixel 5). One amazing project to follow here is the Vicuna project, which is performing research in making these models more performant and portable. Databricks is also doing incredible work here with their Dolly project.

I too was initially skeptical about how capabilities like ChatGPT could be brought to my customers in the Defense and Intelligence Communities of the United States. Now, I'm more confident than ever that LLM providers will begin to feel the pressure of delivering these enhancements so organizations can maintain their air-gapped data privacy while still advancing AI capabilities.

What About 2025?

I won't even try to predict what AI could look like in 2025. We are seeing exponential advancement in this space and new tools/paradigms could emerge that push that growth even more.

As organizations and individuals adapt to these changes, it will be crucial to understand the implications of AI development and how to harness the power of LLMs effectively. By staying informed and prepared, businesses can leverage the advancements in AI to create powerful solutions and reshape industries. The future of AI is bright, and those who embrace these changes and adapt to the new reality will undoubtedly thrive in this dynamic ecosystem.

While these tools have done a lot to help with software operations, they are constantly evolving and forming new architectures and frameworks as new technology is introduced.

I was lucky to join Harness, the Modern Software Delivery Platform, in 2022. At Harness, we are laser-focused not just on enabling customers to eloquently and efficiently tackle all aspects of the Software Delivery Lifecycle (SDLC), but we are thinking ahead to the future and identifying the paradigms and practices that are going to revolutionize how our customers conduct their software operations and developer experiences.

These are the top new DevOps paradigms and practices to look out for in 2023, as well as what we’re doing at Harness to enable our customers to adopt and excel at them!

Putting the Sec into DevSecOps

DevSecOps is a relatively new concept that has been gaining traction since the mid-2010s. It is a response to the movement of DevOps, which began in 2009. Short for "Development, Security, Operations," DevSecOps is an approach to software development and security that seeks to bring security earlier into the software development cycle. This is achieved by integrating development and security practices into the DevOps culture. The goal is to create applications that are secure by design and are managed, monitored, and tested in an automated and continuous way.

Initially, Security was siloed from DevOps, with various scans and security tests performed by a distinct security team after code was built and/or software was deployed. As DevSecOps has evolved and gained adoption, this approach is no longer feasible. To stay ahead of the curve, organizations will need to begin to shift their focus "left” to embedding security into the DevOps culture.

Shift left security is an approach to security that focuses on embedding security earlier in the development process rather than waiting until the end of the process. It also emphasizes the need for collaboration among teams such as developers, operations, and security to detect and manage security risks throughout the entire software development life cycle. By taking a shift left approach, organizations can implement comprehensive and automated security measures that are tailored to the specific needs of their organization, applications, and environment. This helps them to identify and address security vulnerabilities promptly, reducing the risk of a costly security breach or data loss.

Harness is leading the charge in DevSecOps with our Security Test Orchestration (STO) module, by providing customers with automated vulnerability assessment and patching solutions integrated directly into their software delivery pipelines. By leveraging machine learning and continuous discovery, customers can ensure that the security policies they defined are continuously enforced, and patches are automatically deployed without affecting operations.

Read more about Security Testing Orchestration at Harness.

Separate Releases from Deployments

While the SDLC has improved immensely since the days that software was distributed primarily by floppy disks, releasing features is still a major point of contention and friction between Developers, Product Managers, and IT Operations staff since the release of features is tied to the deployment of software.

First, the release process can be slow and fragmented. This can be due to a manual approval process, lack of automation, or simply legacy tools that are not built for speed and agility.

Second, the lack of control over the release process often leads to frustration among feature stakeholders, as they have no insight into what is happening and can't control when the release will be live.

The inability to decouple releases from deployments can also have implications for the software itself. For example, if multiple changes are released all at once, it can be difficult to isolate a regression bug or identify which release/deployment caused an issue.

A better approach to releasing software is to separate the release process from the deployment process. This is done today primarily using feature flags.

A feature flag (also referred to as a feature toggle, feature switch, feature bit, feature control, dark launch, mode flag, or feature flipper) is a technique in software development that attempts to provide an alternative to maintaining multiple code branches for features that are under development. Feature flags allow developers to modify system behavior without making hard code or configuration changes. Feature flags can be used to control the release of features to selected users, or to enable the gradual rollout of features over time. This can be used to test the impact that a new feature might have in production

Feature flags also allow for quicker and more comprehensive testing on the release side before the deployment is executed. This also allows for more granular releases, giving stakeholders more control over the process. Furthermore, it gives developers the ability to roll back a release without needing to redeploy the software.

With Harness, customers can separate releases from deployments by doing feature flagging and A/B testing directly in their DevOps pipelines using our Feature Flags (FF) module. This ensures that teams can deploy and test software without interrupting operations and without any extra complexity.

This also allows developers and product managers to take advantage of many of the other innovative paradigms that Harness provides for their feature flag strategies, such as having policy-enforced, pipelined rollout strategies and responding to the results of Continuous Verification through our Site Reliability Management (SRM) module.

See how Harness does Feature Flags and assess your organization's Feature Management Maturity.

Deploy Hourly

Agile software development methodologies have done fantastic things for the SDLC by helping take software deployment frequency from months down to the size of a single “sprint,” often one or two weeks. Releasing on sprint-based Agile cycles has been a standard for over a decade, and has made sense, as code changes usually had to go through hours of unit testing and regression testing, hours of security tests, and hours of strenuous deployment execution and monitoring to be considered successful.

However, in today’s increasingly competitive business landscape, deploying once every week can be far too slow. As far back as 2015, Amazon was conducting over 130,000 deployments every day. (The Phoenix Project)

Businesses need to operate at the speed of the cloud. To stay competitive, organizations must be able to move faster than their competitors. This is especially true in markets that experience rapid change, such as the world of e-commerce or SaaS. As a result, organizations must look for ways to reduce the time it takes to develop and release products.

The Harness Continuous Integration and Continuous Delivery include features like:

• CI Test Intelligence that reduces your unit testing from hours to minutes using Machine Learning and code analysis.

• Automated security testing orchestration.

• Intelligent pipeline stopping and deployment rollbacks based on:

  • Unit test results

  • Security test results

  • Site Reliability thresholds

  • Many more options

By automating the release management process and eliminating manual activities, organizations can deploy as rapidly as hourly, ensuring faster delivery of features and fixes.

Read a bit about how Harness is the fast CI on the planet.

Automated Rollbacks

In current DevOps operations, everyone involved does absolutely everything they can to avoid the dreaded rollback. CI/CD rollbacks can be painful for teams due to the complexity and time involved in manually reverting code to an earlier version. This can be especially difficult if the code includes a lot of changes that have been made over a long period (see “Deploy Hourly” to see how to mitigate this). Additionally, the process of reverting to an earlier version can be difficult to track and document and can lead to unexpected results. Reverting to an earlier version can sometimes cause issues with the application, such as conflicts in libraries or configuration settings.

With Harness CD we provide templated and easy approaches for industry-standard rollout methods for modern deployment targets, including Canary and Blue/Green. For each of these deployments, we also include a templated rollback that provably works every time, giving your team the confidence to deploy more frequently without fear of the dreaded rollback.

With Harness, teams can set up automation to the point where CI/CD pipelines can seamlessly roll back deployments in the event of an issue or regression. Additionally, there are several safety catches built in that will ensure that a rollback will only happen if the necessary conditions are met. This can be particularly beneficial for organizations that need to move quickly and avoid manual intervention and delays in the deployment process.

Harness also allows for a detailed audit trail of every release, including snapshots of what code was deployed and when.

See how our customers are reducing rollback and deployment time for their operations.

Comprehensive Governance

At this point, you’re probably saying, “all of this stuff sounds like it will be great… if I can get developers to actually use it!”

Governance for DevSecOps is always a challenge, usually defaulting to two approaches:

• Wild West - everyone on the DevOps team can create new pipelines, register connectors, write and use secrets, etc. This may lend to some great velocity, but quality, security, and governance all degrade as new organizational requirements emerge for these pipelines.

• Lockdown - A few people in the organization hold the “keys to the kingdom.” This might lead to better quality and security for CI/CD pipelines, but organizations take a major hit to velocity as these individuals are single points of failure and chokepoints for modifications.

Harness provides DevOps teams with the ability to adopt the best of both approaches without tradeoffs: teams can maintain great velocity while proactively addressing security and governance requirements.

Harness provides role-based access control (RBAC) to ensure that teams have granular access to the right components. RBAC allows teams to grant or deny access to individual resources, such as pipelines, secrets, and connectors, as needed. Additionally, it allows for pre-defined templates for pipelines, secrets, and more that can be shared.

All of this can be governed with our first-class implementation of the Open Policy Agent (OPA), integrated into Harness as a policy enforcement engine. This engine allows AppSec and SecOps teams to define and enforce policies that control how applications and services are deployed. The OPA is a powerful addition to the Harness platform, allowing teams to assert an even greater level of control over their CI/CD pipelines and deployment strategies.

Underlying Harness pipelines is GitOps. The GitOps workflow is based on the idea that your Git repository is the source of truth for the state of the pipeline your organization is deploying. Every time any change is made to the pipeline state, it is pushed to the repository. This combination of version control and automation ensures that no change is ever made outside of the Git repository, allowing for total visibility into the state of the pipeline at any point in time. Updating pipelines is as simple as updating the original templates repositories they were branched from.

Harness also provides comprehensive change management and audit control. This allows teams to better control changes to pipelines, as well as perform post-deployment analysis to ensure that the desired outcome is reached. By tracking changes made to pipelines, release cycles, and configurations, it becomes easier to identify issues promptly and reduce the impact of any issues that do arise.

See more about how Harness is providing first-class Governance without the headache.

Conclusion

There are a lot of exciting DevOps paradigms to consider and adopt within your organization in 2023. With Harness, we are always looking to the feature to stay one step ahead of the needs of our customers and we look forward to seeing these paradigms play out.

This article reflects my personal excitement for products I happen to sell and does not necessarily reflect the opinions of my employers. In other words: my opinions are my own.

I was inspired by two sources to write this narrative:

• Investments Unlimited by Beal et.al. - This was an excellent and relatively short book providing the importance of standardized DevSecOps practices in the form of a narrative much like The Phoenix Project

• What 2026 looks like by Daniel Kokotajlo

I think stories and narratives have a place in tech writing and can convey a point or paradigm more powerfully than any kind of more objective writing. I hope this gives you an idea of where I think AI could get implemented into our lives in the near future.

I hope you all enjoy it!

"Emma, why don't you tell the class a little bit about your narrative piece and how you wrote it?"

Emma looked up from her computer in response to Professor Smith's question. Admittedly, she was distracted. She was reading up on the latest LitGPT4 model coming out of OpenAI and wondering if it was worth swapping out the model in her current workflow from Google's LitGen.

"Of course Professor! Give me a second to share my screen and I'll walk through my process" she exclaimed.

Emma quickly proceeded to the front of the class and shared her presentation to the projector through the school's casting service. She began to explain her narrative piece, taking the class step by step through her writing process. Her writing tools were pretty simple: Microsoft Word was up on the left of her screen but to the right were a few AI-enabled autocompletion tools with various toggles and prompt windows.

"So, for this piece, I knew I wanted to write something from the perspective of a young kid facing adversity. New York City has been a major American literary and cultural Mecca so I decided to start with a LitGen prompt of 'young kid living in 1970s New York City has to find a job, Ernest Hemingway, 20 pages.' What came out was pretty impressive but the tones mismatched the period since Hemingway was really writing a majority of his work about 50 years earlier."

Emma was a Senior at Carnegie Mellon University's English school, class of 2026. Like all the other students, she was eager to learn and succeed. She had heard of the new technology sweeping the world - Artificial Intelligence (AI) - and was determined to learn how to use it.

At first, Emma thought it would be an easy task. After all, she had been studying English for years; surely AI would just be a few more tools in her toolbox. She soon discovered that wasn't quite the case.

"What really stood out to you about the story produced by the AI?" Professor Smith asked.

"Well," Emma pondered, "I suppose the most interesting thing to me was that Litgen defaulted to a rich white Christian kid living in Manhattan as the subject of the narrative. The subject also spoke in a tone that lacked any kind of vernacular or colloquial language for the time and era. I thought this likely stemmed from a heavy initial model weight for Hemingway as the author and my model output analysis basically confirmed that."

She highlighted a selection of text referencing the character and a visualization tool opened up to her right. At the center of the visualization was her highlighted text, but extending out from it were various other terms and phrases. The term "Ernest Hemingway" was very bold and big relative to many other words.

She continued, "this was a narrative element I didn't necessarily agree with. Luckily with the context-agreement tool, I just had to tune a few things in the first few paragraphs and I had a more historically agreeable subject consistently applied through the rest of the narrative."

The AI-driven writing software that was emerging was far more sophisticated than anything she had ever seen before. It could analyze complex data, detect subtle nuances of language, and even generate its own creative ideas! Emma quickly realized that if she wanted to stay competitive in this new world of technology, she would have to learn how to use AI for writing.

"Another thing I wanted to tune was the ending. The current implementation of LitGen overwhelmingly outputs positive endings both due to the bias of the dataset as well as the bias of its developers at OpenAI to prevent it from being used for negative misinformation or poor user reactions. As a result, my narrative had a cheery and positive ending where the subject got a very well-paid job quite easily. I didn't think this fit the overall tone and theme I was trying to convey." Emma stated.

"In your final draft I read that didn't seem to be the case, what did you do to fix that?" asked Smith.

When AI-driven writing first started gaining popularity around 2022, professors across the United States initially did not like the usage of AI in their classrooms. Students who were motivated to just get a passing grade were copying their writing prompts directly into tools like GPT3 or ChatGPT and turned in entire AI-written essays. Some companies emerged producing software to find the essays that were written by AI, but open-sourced AI emerged that would just re-write the text based on a randomized number, resulting in a constant game of cat and mouse.

However, as time went on and students began to understand the nuances of AI-driven writing, these same professors began to see the potential of this technology and many of them began to actively encourage its usage in their classrooms. Professor Smith became a major trailblazer among his colleagues in English Literature academia when he introduced "AI-Enabled Fiction Writing", a 101 writing course produced in coordination with the Computer Science department.

Like Data Science classes that were popular as early as 2016, the class demonstrated to students that AI could be tightly integrated into their writing process without much coding or technical knowledge. Thanks to it's popularity, Emma has been able to attend the inaugural follow-on "Advanced AI-Enabled Fiction Writing" in her final semester at CMU.

"Good question, Professor." Emma started excitedly, "I thought this was going to be difficult and I would have to hand-correct it. It turns out that, luckily, Project Gutenberg conducted a huge initiative with Snorkel AI to programmatically label the outcome of all of their books by a selection of outcomes like "good" or "bad". More importantly, they also labeled those outcomes by various sentiments. I just went to their website and pulled all the collated text for all the books labeled 'Bittersweet'."

She began to get excited. She knew what she was about to say next was going to excite the class.

Emma smiled and continued, "I used the sentiment-tagged text from Project Gutenberg and enlisted the help of a friend from the Computer Science department to help me with this last part. OpenAI provides guidance for a method they call transfer learning of LitGen. That was a little out of my technical wheelhouse, but my CS friend was able to use my Bittersweet dataset to produce a new and improved LitGen model I've called 'LitGen_Bittersweet.' Re-running the text through the model and prompting for an ending was able to craft something more appropriate to my goals for my narrative. Originally, the subject inexplicably got a job as a successful banker. Now, the subject gets a more basic job as a dishwasher but still pines for the cushier jobs he has observed around the city."

Astonished murmurs rose from those around the class. Professor Smith was impressed with Emma's work and nodded in agreement as she finished her explanation.

"By the way," Emma continued "LitGen_Bittersweet is now available for everyone here through our private Academic OpenAI account. You should all be able to use it for story-tuning going forward if you'd like to incorporate more bittersweet elements into your writing."

"Very impressive Emma! It's amazing to see the progress Emma has made in using AI to enhance her writing," Professor Smith said, "I'm confident that this skill will prove invaluable as she begins her new job after graduation. I think we could all learn a little bit about going the extra mile by tuning our generative models with transfer learning, very impressive collaboration."

Emma grabbed her laptop and sat down back at her desk, beaming with pride.

Professor Smith continued, "Now, let's move on to the next student and see what kind of creative stories we came up with today..."

Be Inspired

The Youtube channel "Answer in Progress" had a fantastic video entitled "why you stopped reading" that really resonated with me. Sabrina, the host, talks about how she used to a be a prolific reader when she was younger, what factors might have caused her to significantly reduce her reading, and runs an experiment of reading 5 books over 5 days with 5 methods to reignite the habit. It's a well-produced watch I recommend for everyone.

This video from videographer/storyteller Max Joseph provides excellent perspective and a little bit of a scare on why you should read more as well.

Make the Decision

The Goodreads 2022 Reading Challenge was my initial decision. Goodreads suggested a goal of getting through 15 books by the end of 2022, which I actually thought was going to be a challenge. I have since set a new challenge to complete 70 books by the end of 2022. You can do this challenge or a similar but I think it's important to set a direction you would like to head in.

Train the Muscle

Fun fact: I spent about 2 years of my life training closely with a bodybuilding coach to lose approximately 80 pounds of fat and double all of my maximum lifts. I learned that when it comes to building a muscle you can get a fantastic coach and follow the greatest gym plan in the world, but at the end of the day the size of the muscle is going to depend on repetitions performed many times over many weeks.

Attempting to increase my reading has been much like a training a muscle to me. Initially, getting through that first book of 2022 involves a lot of strong decisions such as "I am going to get through 30 pages today even if I have to do it right before bed."

Soon, much like going to the gym, it will become second nature. You will start to feel uncomfortable on the rare days you cannot get your 100 pages in. You might even find yourself bingeing entire books in the span of just a couple of days like I began doing fairly consistently.

For example, my reading trends so far for 2022:

• January: 1 Book
• February: 4 Books
• March: 8 Books
• April: 10 Books
• May: 2 Books (as of May 7)

Read Good Books

There's nothing that stops your progress like trying to trudge through a book you hate. I've converged on a kind of "Bloom's Taxonomy" but for finding a good book you will actually enjoy:

1. Recommendations from friends/family/co-workers that know you well.
2. Recommendations from educated strangers online. The r/suggestmeabook subreddit is an incredible place to either ask a community or browse through old threads.
3. Recommendations from librarians or bookshop owners.
4. Books that are popular and/or have very high ratings on Goodreads.
5. Books that just look interesting while you're perusing.

Do not be afraid to give up on a book. My general rule is to shoot for 10% of the book, but if it doesn't draw you in then quit. There will likely be an opportunity to re-approach the book if it's truly worth a read. Several times I mentioned to a friend that I quit a book and they said "oh, trust me it gets really good about halfway through." I picked it back up and they are usually correct.

Physical for Fiction, Kindle for Nonfiction

As you read, you'll start to converge on your systems that you do and don't like. Here are a few that have worked for me and may save you a few experiments:

• For fiction books, I will usually try to grab a hardcover or softcover from the library or a local bookshop. It's not significantly better than a Kindle version of the book but I do slightly prefer it.
• For large, non-fiction, I will usually try to grab an ebook copy. The reason is that these books often use footnotes and endnotes which you can quickly read by clicking on them. If the note is in the end of the book then you will be constantly flipping back and forth to the back of the book to ensure you get all the context which is not an ideal reading experience. In addition, non-fiction books usually have several passages and statistics you want to remember and with Kindle you can highlight passages to store in your notes and share to your Goodreads profile.
• If you have long periods where you get an opportunity to read, I recommend borrowing both the audiobook AND the ebook or physical book. Either of the formats can be hard to pay attention to for long periods of time, but when you read the book while listening to the audiobook at about 2x speed you become a hyper-focused reading machine. Don't knock it until you try it.
• Buy a Kindle when it goes on sale. If you don't like buying books from Jeff Bezos you can rest assured that you can you use it primarily to read ebooks from the library using the Libby App.

Record your Milestones

For just about every book I read, I now write a Goodreads review for it. I do this for two reasons:

1. Depending on the size of the book (Snow Crash by Neal Stephenson was a behemoth), you might have just gone on quite a literary journey that took you longer than an entire season of a TV show to complete. It makes sense to reflect while the details are still fresh in your head.
2. You'll be a great resource for future friends. I've had several situations where a friend told me they liked a book and when I asked why they would reply with "hm... I don't actually know, it was so long ago." I might take their word for it, but it's nice to have the additional context to discuss with them.

Send me a friend or follow request on Goodreads and I look forward to seeing what books you all read this year!

For the general user, while the platform is definitely a great place to keep in touch with colleagues through a simple click of the "Connect" button, LinkedIn falls behind on some opportunities to really explore your network in a comprehensive way. Currently, this functionality resides almost entirely in the Search Bar which, while effective, is a relatively narrow view.

I think this problem provides an excellent opportunity for those interested in BI and visualization to get started in Qlik Sense, one of the largest enterprise BI platforms. What better way to learn than to explore data that is directly beneficial to you? Your professional network is one of your most prized assets and using tools to really explore and understand it can get you far.

With all that said, let's get started!

1. Get Your Qlik Sense Account

While anyone can get a free 30 day trial license, I will call out that students get a full calendar year of the Qlik Sense platform with their .edu login.

Either way, log into the Qlik Sense platform and you should be presented with an interface like this:

2. Get Your LinkedIn Data

In order to pull the data we need live and programmatically, we would need to utilize the LinkedIn Connections API. While this could be an extension to the project in the future, for this example we are going to go a much easier route that doesn't require understanding of more complex ETL through APIs/OAuth and just grab simple flat data exports directly from LinkedIn.

First, go to your "Settings & Privacy" for your profile. Then, select the following to request a CSV download link sent to your email. In this case we're only interested in our "Connections" but you can certainly request and perform interesting visualizations for just about every other piece of LinkedIn data you have.

After around 10 minutes you should get the link and have your CSVs ready to go.

3. Create Your Qlik Sense App

Load Your Data

While there's many ways to start this project, we're going to simply start with a "New analytics app."

After selecting the "Add data from files and other sources" box in the middle we are presented with a space where we can simply drag and drop our "Connections.csv" file from our LinkedIn Data download.

The data export is admittedly a little odd so we'll need to configure some things before Qlik recognizes the schema of the file.

1. We'll need to ensure Delimiter is set to "Comma"
2. The headers are actually 4 rows down from from the top of the sheet, so we'll need to indicate that as well.
3. While you don't have to do this, you can use this step to change the current column headers from the numbered labeling to ones that are more recognizable.

Then just hit the green button in the bottom right and your data is loaded!

Explore Your Data

Now that our data is loaded in we are presented with our application and we can start to dive into it. Go to the "Sheets" at the top of the screen, this will be the home of our charts, tables, and visualizations.

Traditional data visualization and exploration best-practices demonstrate that we should start with the highest level that we can first. Let's try to get a view of all the companies that I have contacts at.

The best way that I could think of is with a tree map. This is a very popular view for showing magnitudes associated with hierarchies. This will make more sense in a second.

We simply pull the tree map chart into our sheet to begin crafting it. For our Dimension, we select "company" and to indicate the number of connections we have at that company we'll just use "Count(first_name)" as our measure. This all translates to "what is the count, or number of, employees each company within the scope of my network?"

For the purposes of this demonstration and to protect the privacy of my network I have actually hashed each person's name and company to hide their details while still retaining functionality.

Great! Now we have a pretty comprehensive tree map of the density of my network at various companies. From here we can start to play with the appearance of this chart in order to make it more useful, such as using a divergent gradient or class of colors to show the size of our companies as opposed to just one single color.

Answer Questions

Now we have a top level view of our data but ultimately what would we want to use this for?

The beauty of platforms like Qlik Sense stems from our ability to start with a top-level view and then dive deeper depending on the goal we're trying to achieve or question we're trying to answer.

Let's suppose you're trying to find your first job, you would likely want to start where your network already has recruiters as they will be able to get a look at your resume and get you past the resume robo-screeners (these are real, they're called Applicant Tracking Systems). When I add tables for the name and role of employees I can now filter through the "Roles" for recruiters. I now have a view of every company where I have a direct connection into their company recruiting pipeline.

When I want to remove this filter, I simply click the big red X, hit "Esc", or remove it as a filter from the top left of my sheet view.

4. What's Next?

Now that you have a baseline to explore your network, there's definitely other ways you can progress.

Cross-walk with Other Datasets

Anyone can re-create what I just did in desktop tools like Excel. Where you really need to take advantage of platforms like Qlik Sense is the ability to model, correlate, and cross-walk between other datasets and APIs. After a little data cleaning, you could likely combine other data from companies like Crunchbase to get more in-depth analysis into questions like "how fast are the companies within my network growing?" or "what are the strongest/weakest overarching industries within my network?"

These are all exercises for another article.

Tell a Story

Lets say you actually want to use data like LinkedIn data to demonstrate to your Chief of Staff that there are distinct centers of excellence for your company that you are not recruiting from. Their time is immensely valuable, you can't waste it filtering and navigating through your visualizations. Qlik Storytelling is an emerging feature of the product that allows you to create presentations that will dynamically pull up pre-filtered views of your Qlik sheets in a slide-by-slide Powerpoint fashion.

Expand

With good, clean data and well-organized schemas, modern analytics tools can do amazing things. With some testing, your data can feed Qlik's Insight Advisor which allows you to ask human-language queries like "how many people work at Microsoft?" and get back clear answers and visualizations on the fly!

I’ve strategically navigated and touched most major enterprise toolsets for managing Agile projects and driving product strategy and I’ve compiled a list of platforms and tools that are:

1. Some of my favorites and

2. Very friendly to students who still have access to their .edu email.

Like before, many of these will even have friendly free-tiers regardless of your student status.

If you are a student, my recommendation is to get out of Word and Google Docs and pick tools to start using them for every aspect of your life in university:

• Manage your grades and classes

• Manage your group projects, especially your capstones projects

• Manage your student organization. These are toolsets I wished I utilized during my time as both Secretary and Recruitment Chairman for my fraternity.

While there may be initial growing pains and a learning curve, you will be distinguishing yourself from other grads by demonstrating that you are already comfortable operating in PM toolsets that many of the biggest enterprise companies are currently utilizing. It also allows you to exercise valuable skillsets around Product and Project Management and use frameworks that are not often explicitly taught in University settings.

Atlassian

Atlassian is the probably the biggest player in software management. It is trusted by major companies and brands like Dropbox and Visa. Their products also act as the backbone for DI2E, the platform allowing modern collaboration and development within the United States defense and intelligence communities.

Some of their top products include:

• Jira — Break your project release down into agile components like Epics, Stories, and Tasks that you can then assign and track across your team. Through their built-in automation suite and Jira Query Language (JQL) you can build many clever automations such as emailed reports of tasks TODOs tailored to each group member.

• Confluence — A wiki tool that allows you to write documentation around your projects, take meeting minutes, and much more. With a powerful indexed search functionality, you don’t need to go digging through mountains of documents to find your notes from that one meeting with a project customer.

• Bitbucket — A git interface that allows you to collaborate on a singular code-base for software-driven projects.

Just about all tools in Atlassian are free for up to 10 users and a limited but still usable amount of storage space. This should be enough for capstone or group projects at most Universities.

Even if you decide not to use their products, Atlassian’s open Team Playbook is nothing to scoff at! This page acts as an encyclopedia of industry-tested “plays” or instructions for guided sessions for your team to do everything a distracted or failing group project could possible need to get back on track!

If you want a quick primer on Agile and Jira, I found the “Agile with Atlassian Jira” on Coursera to be an excellent introduction to both!

Asana

Asana seeks to be the home for all productivity and creativity at your organization and they are trusted at major enterprises like Deloitte and Google. They have too many solutions to name with hundreds of proven templates and workflows around:

• Product Strategy

• Business Processes and Automation

• Agile & Scrum Project Management

• Customer Relationship Management

• many more!

While Asana does seem to be a little weaker on software-specific project management, it plugs the holes by offering integrations with other platforms like Github.

If nothing else, the tool is worth exploring in order to remove some of the complexities of project development by leaning on their templates and frameworks to help you and your team take a project from initial design to deployment and maintenance.

Asana has a strong free-tier and Asana Premium is free for all students.

Azure DevOps

Often coming with Microsoft and Azure enterprise license packages, Azure DevOps is often the de-facto tool found at many companies and Universities.

Azure DevOps offers all the same tooling that can be expected from software and project productivity suites:

• Azure Boards — Split your work into Epics, Stories, and Tasks and track them across Kanban boards for your team.

• Azure Repos — Private git repos for maintaining your team’s codebase.

• Extensions — Azure DevOps has a strong library of extension allowing you to connect your project in clever ways to other tools like Microsoft Teams or Github.

While you could use this suite to manage just about any project, Azure DevOps is definitely focused on software releases with a litany of integrations into tools like VSCode and Azure to provide a singular interface into application development and performance.

If your school relies on Microsoft as their primary office suite then it is very likely that you will get free, unlimited access to this toolset with your student license. Contact or browse your school’s IT resources for more information. If your school can’t help you then Microsoft Azure will give you access to the tools for free along with $100 Azure credit if you have an .edu address.

Basecamp

Unlike the other tools in this list, which seek to address management for an individual project and operate on a much smaller project-team scale, Basecamp seeks to address issues around communication and management for an entire company or organization. These issues are often found when relying on email or static file attachments. Basecamp is an excellent spot for managing a collection of project teams, perhaps a community of students in your particular school (Computer Engineering class of 2021) or a place for your student organization.

Every project within an organization’s Basecamp has 6 offerings:

• To-Dos — tasks to be completed with assignments and due-dates

• Message Board — more persistent, forum-style discussions and announcements

• Project Schedule — A calendar of dates to hit with integrations for all major calendar apps

• Docs & Files — a location for individuals to collaborate and show off documents and assets

• Campfire — a simple group chatroom

• Check-ins — recurring questions for all within your project

While Basecamp also has a strong free-tier, they offer free Basecamp Pro for Classroom for teachers and students. Basecamp is definitely a strong offering for anyone who wants to create a greater sense of community.

Honorable Mentions

Airtable

It’s probably easier to say what Airtable is not because this platform is packed with so many templates that it’s hard to think of what you can’t do to manage your project:

• Gantt charts for timelines

• Calendars

• Bug Trackers

• CRMs

• and many more!

With a strong free-tier allowing for over 1,000 records and 2GB of attachment space it’s worth a look for aspiring project and product managers. If you’re a student, they are offering free access to Airtable Pro for up to 2 years as well as a wealth of ideas for how to use the platform for personal organization!

Notion

Notion is my personal favorite “notebook” application, but it is so much more powerful than that. Through clever combinations of their tooling and ability to use templates created by them and other users, the platform is similar to Airtable in that it can be molded to fit just about any use-case you can imagine.

Notion offers free Notion Pro for anyone with a .edu email address, which includes unlimited file uploads and unlimited guests collaborating, an incredibly generous offering for such a versatile platform!

*This is a cross-post from my LinkedIn.*

Congratulations upcoming grads! You’ve sat through countless hours of classes, written your sorting algorithms from scratch, analyzed your lab data in Excel, and you’re about to receive that piece of paper stating that maybe you know a thing or two about your STEM fields.

Regardless of your field, I don’t need to tell you that with the new volume, veracity, velocity, and availability of data today just about every industry is integrating data collection and analytics into everything that they do.

This is happening even in fields that are outside of the cloud compute giants! Civil Engineers going into construction will need to learn to use data from previous contracts to forecast risk and make financial projections for a customer. Cars are becoming computers on wheels and are producing more data and telemetry than ever before. Car designers can use this data to improve future models and make preventative maintenance predictions. Just about every major U.S. government body and defense/intelligence agency that you could possibly work for has released new strategies stating that data and analytics are top priorities for staffing and training moving forward.

If you’re like me, right after I graduated, then you’re probably thinking “I’ve just been using Excel to analyze maybe a few thousand rows for the past few years, how the heck do I get started with all this crazy data stuff?”

Have no fear! I’ve strategically navigated and touched just about every major enterprise analytics tool available during both my professional and graduate studies and I’ve compiled a list of platforms that are:

1. Some of my favorites, and

2. Very friendly to students who still have access to their .edu email.

Some of them even have very friendly free-tiers regardless of your student status.

Qlik

Qlik provides end-to-end data analytics solutions for huge enterprise customers like Cisco, Samsung, and Deloitte. I personally have used the tool and it will likely meet 99% of your data analytics needs for your organization. With a comprehensive licensing model and scalable backends, it’s not unusual to see the suite at a lot of major enterprise companies and Federal agencies. Even if your future employer isn’t using Qlik-specific products, the skills and foundational knowledge you would get from building on the platform will likely translate across other analytics platforms as well.

Currently they have an incredibly permissive Qlik Academic Program that is targeted at students and educators. The program offers one year of:

• Access to the Qlik Continuous Classroom which provides education on the Qlik platform as well as foundational knowledge for anyone to get started in data and analytics.

• Free access to their enterprise Qlik Sense platform to practice your data ETL and visualizations, all in your browser! No download or installation required.

• Pathways to get Certified and Qualified on their Qlik products and foundations of data and analytics. These are valuable, industry-recognized milestones to provide new trust in your resume.

If you’re looking for a place to get started in enterprise-sized Business Intelligence and analytics, it’s hard to beat Qlik!

Databricks

Databricks is widely recognized as the go-to platform for data science and machine learning on the biggest and most complex datasets. Through a combination of the Apache Spark engine and distributed high-performance compute clusters, users can run ETL and analysis jobs in seconds to minutes that would usually take hours to days. It’s no wonder the platform is trusted by major clients like Shell and Nielsen.

Currently, they offer the Databricks University Alliance program which is one of the most generous offerings I’ve seen from platforms operating in these domains. Signing up with your .edu email gets you:

• Free access to the Databricks Community Edition: a sandbox space that should allow you to exercise just about all functionality in the platform.

• Self-paced Databricks courses. This is the most generous part of the offering, as these courses would otherwise cost thousands of dollars if you were purchasing them as a company or independent party.

The Databricks platform allows you to interact and push compute jobs to your clusters using Scala (a higher-level Java derivative), Python, pure SQL, and even R! It is the perfect way to get started all the way from the basics of Data Science all the way to Machine Learning operations on big datasets.

Tableau

Another major player in Business Intelligence and Visualization is Tableau. Where platforms like Qlik may fall behind on visualization, Tableau has been the king of this domain for over a decade. The platform is a trusted BI component for many big players like Verizon and Charles Schwab. If you need a beautiful, awesome visualization or interface then Tableau is the platform for you.

Currently they are offering through their Tableau for Students program:

• A free one-year license to their power Tableau Desktop and Tableau Prep software.

• Unlimited access to the Tableau Public platform (a lot of the same functionality, only your visualizations have to be public on the web).

• Comprehensive free at-home learning resources.

• 20% off the cost to take certification exams.

If your interests lie primarily in crafting beautiful data visualizations then Tableau is the platform for you!

Power BI

Microsoft has its own BI and visualization platform, PowerBI, for customers who really lean on the company. PowerBI boasts proven patterns for BI and visualization of data derived from other Microsoft applications and platforms such as Azure and Dynamic CRM. It’s the preferred BI platform for companies like Adobe and Nokia.

• Microsoft has a permissible model even if you’re not a student! The only catch is that the browser version of Power BI Pro is $10 a month while Power BI desktop is free… if you’re running a Windows PC.

• Microsoft also maintains a wealth of learning material including free material to help you achieve the DA-100 Certified: Data Analyst Associate certificate.

PowerBI is a strong, entirely free option that’s well-respected in the industry for learning the ropes around data analysis.

Colaboratory

I don’t think Google needs an introduction; it should come as no surprise that they are an industry leader in Data Science and Machine Learning. Only a few years ago they productized their internal Jupyter Notebook platform and made it available for public release as Colaboratory.

Regardless of your status as a student, Colaboratory is available for free usage to anyone with a Google account, in a model similar to Google Docs. It offers one-click deployment of Jupyter python notebooks run and hosted on their servers, no credit card required!

• Connection to GPU/TPU runtimes

• Connect to your Google Drive to work on larger datasets

• In-app search library for popular code snippets

• Publish your Colaboratory apps to Github to show them off in one-click deployments

• Much more!

Colaboratory lowers the bar immensely for getting started with Python-based data science and machine learning by providing reproducible environments to use and exercise tools and frameworks like Tensorflow, Keras, and Scikit-Learn!

Deepnote

One of the newest entries to the Data Science and Machine Learning set of tools is Deepnote.

Deepnote hopes to be the data science platform for you and your team, offering some of the most comprehensive workflows I’ve seen in the industry. While the company is still growing, they offer one of the most generous free-tiers I have ever seen in a platform. You can even run your free-tier compute for over 700 hours/month, making it effectively free to run a basic python application indefinitely!

• Upload your own data or use some of their pre-built connectors for popular datastores like S3 or MongoDB.

• Run your own Docker container environments pulled from any popular image registry. For those who don’t understand what that means, learn a little Docker 101.

• Connect your project to a Github repository to push/pull proper versioning.

• Invite collaborators who can modify and comment on your shared work!

While Deepnote is a smaller, new start to the Data Science community, I am very impressed with what they’ve built and encourage everyone to check them out!

This is a cross-post from my LinkedIn. Read and connect with me on there: https://www.linkedin.com/pulse/data-analytics-resources-students-recent-grads-nic-acton/

In this scenario, we are considering an anomalous application; imagine a Netflix or Facebook if it makes it simpler. We are equating it to the construction of a small dollhouse. To extend the analogy even further, we'll say that the dollhouse itself is the infrastructure that the application lives on and the dolls and furniture are the application code itself (makes sense, those are the parts that the users actually want to interact with).

On-Premise

On-premise infrastructure would be similar to building the dollhouse by yourself. You design every piece of it using your experience with dollhouses, you run to the hobby store and buy every piece, you put it all together, and then you give it to the users who then put in their furniture and dolls.

There are definitely benefits to the approach:

1. Once you're done building the dollhouse, you're done. You don't have to buy any more things or make any more runs to the store, nor do you have to keep paying for what you have.
2. You (and maybe a partner or two) built this dollhouse, so you know and understand it inside and out.

However, you have to recognize the negatives:

1. What happens if a piece or whole room breaks? You now have to run to the store and get new parts.
2. What happens if dollhouses get newer and better? You will have to keep upgrading your house to keep up.
3. What happens if the person who helped you build it takes a long vacation? Will you be able to fix or understand the parts of the dollhouse that they built?
4. If your users grow out of dollhouses, you're still stuck with the dollhouse until you put in the effort to get rid of it (hope you have a way to do so responsibly and sustainably).

Cloud Infrastructure

Cloud Infrastructure would be similar to leasing a dollhouse from a toy rental company. Bear with me here, I know this sounds odd.

Benefits of this approach:

1. You don't have to build the dollhouse at all, you can just buy it, have it delivered right to you, and the users can start playing with it immediately.
2. If your dollhouse breaks, you just return the dollhouse and they'll give you a new one.
3. If there's a newer/shinier dollhouse then you can give your old dollhouse back and just get a newer one for a premium.
4. If your users grow out of dollhouses and don't want them anymore, you just stop paying.

Negatives:

1. While the dollhouse rental company has a ton of options that cover 99% of dollhouse needs, if you don't like some granular specifics about your dollhouse then you basically have to request changes to the dollhouse company and hope there's enough interest for them to include it in future offerings.
2. The dollhouse company is not cheap. Depending on the amount of time you need the dollhouse for, it could have been cheaper if you went with the first option and built the dollhouse yourself.
3. If the dollhouse rental company goes out of business, you no longer have dollhouses.

Cloud Services & Serverless

Cloud Services and Serverless offerings would be like Legos. If your user wants space for their dolls and furniture, they can use the legos to build a house.

Benefits:

1. Your users build want they want. If they want a simple little house that meets 90% of what they want then they can do that in a relatively short amount of time. You can consult them along the way for more complex builds to get that last 10%.
2. If a Lego piece breaks, you can immediately just swap it out with another one. No need to return the whole house.
3. Lego builds pieces that are consistently available and are guaranteed to be best-in-class quality.
4. Lego provides instructions and templates to build basic houses, so everyone can build the exact same house with the exact same pieces.
5. Of course, you can also add more rooms to your lego house pretty easily.
6. If you tear down an old dollhouse, the basic Lego pieces are still useful for building new houses.

Negatives:

1. You're used to building dollhouses out of traditional materials, providing dollhouses of the same quality using Legos will take some major re-learning.
2. What happens if Lego goes out of business, or all of a sudden Mega Bloks becomes the next biggest thing? Unlikely to happen, but if it does you're going to feel really dumb for investing so much in Lego.
3. Lego is really new and not well-understood in the dollhouse community. People have been building traditional dollhouses for years and clearly understand the benefits, costs, and optimizations you can make.
4. Since your children are the builders of the house, you're depending on them to make good instructions if another child wants to build the same house (and let's face it, children don't want to make instructions, they just want to build!)

Like most analogies, it's not 100% aligned or correct but it is useful. Again, welcome any feedback and certainly welcome any other analogies that have helped with IT and Infrastructure communication to non-technical stakeholders!

Cover Photo by edgeeffectmedia.com on Unsplash

Make it easy

Effectively, he details this as the effort to remove as many distractions and barriers as possible to a habit that you want to form. One way I've done this in my personal life is that I leave Clorox Wipes on the fridge so that they're not buried away in a cabinet, which is a motivator to keep the kitchen counters wiped and clean pretty consistently.

I have previously attempted using many different code-driven blogging platforms such as hosting basic HTML and hyperlinking a bunch of pages, using Jekyll hosted on Github pages, even hosting Hugo on Firebase . I have learned a TON and I do love the control over look and feel.

All that being said, I was inspired by this post to take a look back at this habit and ask myself "am I distracting myself from what I really want to accomplish, which is more creation?"

Hugo Workflow

Keep in mind, this workflow was actually an improvement from previous workflows in terms of removing the distractions from writing and getting it on the Internet.

1. Open up iTerm
2. Navigate to my blog folder
3. Create a new formatted blog post via `hugo new blog/<post_name>.md
4. Open that new Markdown file in Typora (fantastic Markdown editor btw and will probably be my continued go-to for distraction-free offline editing).
5. If I wanted to add an image, I needed to make sure it was in the static folder and had the correct relative path.
6. Write my words
7. Run a hugo server and jump into localhost on the browser to make sure nothing broke.
8. Build the blog with hugo
9. Make any fixes, including sometimes having to dive into the Hugo theme CSS to fix stuff.
10. Deploy to Firebase
11. Wait for the Firebase changes to resolve and check out the post on nicacton.com

Hashnode Workflow

1. Navigate to Hashnode
2. Hit "Create" then "Write Story"
3. I'm writing right now. Images, tags, links, etc are covered by internal tooling.

I'm going to attempt this same post on Dev.to, another writing platform I've enjoyed consuming, and see how I feel and where the post goes.

Ultimately, I enjoyed the granularity and control of editing the actual website code and having complete control over the look and feel, but I think it's time for me to focus less on that and more on the actual content and consistency of my writing by removing barriers.

Cover Photo by Jakub Dziubak on Unsplash