Microsoft Journal

A Windows experience offering delightful freeform personal journaling built – from the ground up – for touch and pen

Journal hero image, screenshot of the Journal app with handwritten notes and drawn diagrams
"Journal feels absolutely amazing. It's intuitive, easy to use, and extremely easy to get into flow writing with this. All of the text gestures are intuitive and allow me to keep focused on the task at hand. Bravo, Journal team!"
Review from Michael in the Microsoft Store, April 2022

About Microsoft Journal

Microsoft Journal was built, from the ground up, to be a delightful personal journaling experience on Windows using pen and touch. It delivers new AI, intuitive gestures, and connected experiences for Microsoft 365 (subscription required, sold separately). It’s designed for people who thrive when writing out their thoughts and plans; and invites people who love to journal to pick up their digital pen, express themselves quickly, and evolve their ideas.

Key Features:

  • Designed for pen and touch for those who take notes with digital ink
  • A page-based canvas optimized for tablet and 2-in-1 devices
  • New intuitive Ink Gestures to erase and select ink that don’t require mode switches
  • Use touch to scroll pages, or tap ink to select words, sentences, and more
  • Drag and drop selected content between pages, or to your favorite applications
  • Microsoft 365 integration (with M365 subscription, sold separately) to access your Calendar for faster meeting notes
  • Import and markup PDF and images
  • Search and recall using keywords or filters
  • Customize each page with page styles (graph, lined, dotted, music staff, blank, and storyboard)
  • Support for writing in different languages
  • Tactile pen support for ink-feel and ink gestures on supported devices

Video

A look inside: Microsoft Journal, a Microsoft Garage project Journal, a Microsoft Garage project, is an app for Windows that invites people who love to journal to pick up their digital pen, express themselves quickly, and evolve their ideas. The Microsoft Garage is an outlet for experimental projects for you to try. Try it out: https://aka.ms/GetJournal Learn more: https://aka.ms/Journal #TimetoJournal

Journey

Microsoft Journal began with a simple but large question, what would a personal journaling experience be like if it was designed to be used with a digital pen and touch? No mouse or keyboard.

After a quick prototype showed value, the team asked the second big question, “What experiences become possible if software can recognize when I write a star beside a line of ink?” Or, more basically, what happens if an application has a semantic understanding of the way that people naturally write notes. Once the team built a very simple proof of concept that proved machine learning could delightfully detect a star and associate it with nearby writing, the incubation began in earnest.

These basic questions have driven the design of Journal. You can see their influence in the page-based canvas that feels so familiar, in the AI used to enable intuitive editing gestures like scratch-out to erase and instant lasso to select, and in the way Journal recognizes the methods people use to organize their content like stars, headings, lists, diagrams, etc. This all combines in an experience that lets people fluidly create and seamlessly edit.

The Applied Sciences Group (ASG) is an applied R&D team dedicated to creating the next generation of computer interaction technologies and experiences. The team is purpose built to explore a problem space like inking on Windows tablets and produce incubations like Microsoft Journal.

I am so proud of this team. They approached notetaking from first principles and incubated an innovative and delightful experience that advances the state of the art of digital ink. Microsoft Journal uses AI to make sense of ink and understand the natural structure of your handwritten notes. It is a pioneer of the next generation of products that leverage computer aided reasoning to help people create and achieve in ways that have never been possible.– Stevie Bathiche, Technical Fellow and leader of Applied Sciences, Microsoft

During their journey building and refining Journal, the team overcame challenges together across the breadth of artificial intelligence, UX design, software engineering, and hardware.

“Collecting unbiased data from around the world to train AI is normally difficult. Doing it just as the world locked down over the COVID-19 pandemic was a huge challenge,” said Jacques Chamberland, Principal Program Manager. “The only solution was to be patient and persevere.”

Adapting their machine learning models, built from snippets of annotated training data, to the real world required many months of refinement. Managing the performance of Journal so that the entire application can run locally on most Windows tablets was also a significant challenge.

ASG’s incubations are usually shared internally within Microsoft in collaboration with other teams to help Microsoft better understand the opportunities, challenges, and technical solutions in a given space.

Journal is different because it was designed for a specific audience: people who want to use their Windows devices primarily with touch and pen.

Microsoft Journal search experience user interfaceMicrosoft Journal search experience interface

“We decided to share Journal outside Microsoft, so we could learn how it gets used, understand what works and what doesn’t, and incorporate lessons learned back into Journal and share with our partners across Microsoft. The Garage is a great path for teams like ours to get feedback on experiments and incubations like Journal,” said Nathan Luquetta-Fish, Director of New Interface UX. “It’s a great pleasure to see customers using and enjoying Journal.”

The Journal team learned how important it is to intentionally design for digital ink experiences, and that inking is more than just writing content – it’s also about annotating content across various formats like PDFs.

They learned that selecting content easily and intuitively, then editing or manipulating the content, was a hard problem to solve since, unlike text, ink is only loosely structured and people’s editing habits are very personal. But editing was extremely important to get right to improve the flow of the inking experience.

So how did Journal make it to product? The Microsoft Experiences + Devices leadership team reviewed the experience of using Windows with a pen and recognized that Journal could fill a gap. Even though Journal was an incubation (meaning it did not have the full breadth of features of a full product), it still earned a 4.3 rating in the Microsoft store. In addition, lots of positive feedback from Microsoft employees who tried it out was also helpful in showing Journal’s impact.

Microsoft Journal has secured further funding and a dedicated, full-time team to develop and support moving forward. It will be an app that comes in-the-box on Windows pen enabled devices in SV2.

It will also appear in the Pen Menu meaning that people will be able to jump right into Journal every time they undock their pen (on devices that support the pen undock event). Journal icon in pen menu

 

Team

Journal team group photo collage

Journal team, pictured: Brooklyn Littell, Dan Nissenbaum, Gary Caldwell, Badal Yadav, Jacques Chamberland, Monroe Thomas, Prajakta Kulkarni, Oz Solomon, Matt Galloway, Fernando Viton, Nathan Luquetta-Fish, Matthew Stephure

Journal team, not pictured: Oussama Elachqar, Nima Mohajerin, Erich Finkelstein, Sergey Doroshenko, Kyle Krum

Download from: