Every new feature Apple unveiled for iPhone, iPad, Mac, and more this fall (including AI)

Apple unveiled what it calls its new “private intelligence” system beneath the identify Apple Intelligence. The discharge places generative fashions on the coronary heart of the ecosystem of Apple gadgets. 

With Apple Intelligence, your iPhone can prioritize notifications to make sure you get notified solely when it is essential all through your day.

The discharge contains writing instruments that leverage AI, together with rewriting, proofreading, and summarizing textual content options obtainable throughout mail, keynotes, third-party apps, and extra. 

Customers can now create customized pictures within the picture library, together with sketches, illustrations, and animations. This characteristic is obtainable in Messages, Apps, Freeform, Keynote, and Pages. 

Apple Intelligence can faucet into instruments and perform duties in your behalf, similar to “Present me all of the pictures,” “Play the podcast,” or “Pull the information that my coworker shared with me final week.” 

As a result of it is grounded in your private info and context, and might retrieve knowledge from throughout your apps and reference the content material in your display screen, Apple Intelligence is positioned to be your private assistant.

Apple emphasised the security and privateness precautions constructed into Apple Intelligence, notably for on-device intelligence processing. The corporate touted the safety of Apple’s silicon, A17 Professional, and its M household of chips (M1, M2, M3, and M4). 

For duties which might be too massive for on-device processing and should be accomplished within the cloud, Apple unveiled Non-public Cloud Compute, which protects customers’ privateness by working on servers specifically created utilizing Apple Silicon. When customers make requests, Apple Intelligence first assessments on-device functionality, however calls on Non-public Cloud Compute if the duty requires extra energy. Apple reiterated that consumer knowledge isn’t saved or offered to exterior events. 

Siri lastly bought the AI makeover it deserves, first with a brand new look: when tapped, mild wraps across the edges of your display screen. Siri can now higher perceive customers, even when they stutter, as a consequence of extra superior pure language processing (NLP). It now has conversational context, remembering what you simply stated and utilizing it to finish the following activity. Customers also can sort requests to Siri. As a result of it has in-depth product data, Siri can reply questions on performance on iPad, iPhone, and Mac. Siri can even have Apple Intelligence’s on-screen consciousness, permitting it to behave on what it sees. The voice assistant also can take actions throughout apps, together with picture enhancing. With entry to your private context, Siri can perceive and full new instructions, similar to pulling your driver’s license info from a photograph and inputting it right into a kind. The Siri updates are coming to iPad and Mac, too. 

Apple Intelligence additionally powers new options in Mail, together with Rewrite, which gives customers totally different variations of what they’ve already drafted. Options are proven in-line, and Proofread edits for grammar, phrase alternative, and sentence construction. You too can use Summarize to transform your textual content into bullet factors. Good Reply identifies related picks of an e-mail and makes use of them to assist craft a customized message. Summaries will now seem on the high of emails, making looking an inbox simpler. Apple Intelligence may even assist prioritize your emails, putting what’s most essential on the high of your inbox. 

There’s an all-new focus possibility: scale back interruptions. When on this setting, your telephone will solely present you what’s most essential primarily based in your private exercise and context. 

Genmoji permits customers to create AI-generated emojis primarily based on what they sort. You too can create a Genmoji primarily based on a photograph of a good friend. Genmojis could be included in-line in Messages and even used for Tapbacks. 

Picture Playground permits customers to leverage AI on-device to create pictures from textual content prompts, which could be simply shared in iMessage and elsewhere. The characteristic can also be obtainable in Keynote, Pages, and Freeform, and as a stand-alone Picture Playground app. 

Picture Wand within the Notes app transforms a tough sketch into a cultured picture and is obtainable immediately within the software palette. For instance, you possibly can circle a tough sketch in Notes and open Picture Playground to remodel your doodle right into a fully-fledged picture. 

Apple Intelligence can even improve the Pictures app with a brand new clean-up software that removes undesirable objects. Search in movies permits customers to simply discover particular snippets of content material, and customers can create Reminiscences on-demand, utilizing textual content to edit and arrange pictures into films. 

Within the Notes app, customers can document and transcribe audio, which Apple Intelligence will generate a textual content abstract of. This expertise can also be obtainable within the Notes telephone app. 

Apple Intelligence is free on iOS 18, iPadOS 18, and MacOS Sequoia, and shall be obtainable to strive in English solely this summer season. 

iPhone and iPad customers will be capable to customise their dwelling display screen additional by putting apps wherever they’d like on the display screen, versus the standard fastened grid. App icon colours can even be customizable, permitting customers to make apps any coloration they need and even match their dwelling display screen. Customers also can change app icons to darkish mode. 

After 5 years of remaining untouched, the Management Middle obtained a number of upgrades, together with the power to customise its toggles, similar to flashlight, display screen recording, calculator, auto-rotate, display screen mirroring, and extra, by tapping, holding, and rearranging. The Management Middle toggle can even characteristic totally different pages with fully customizable consumer controls, and customers can change controls on the underside of the Lock Display. 

Apple additionally added privateness choices, together with the power to lock an app, which requires customers to authenticate with Face ID or passcode earlier than accessing the app. Customers also can disguise an app, which makes it disappear from the house display screen to a hidden a part of their app library.  

Messages obtained a number of upgrades. Tapbacks, the characteristic permitting customers to react to messages by holding them down, was upgraded to characteristic totally different colours and embrace emojis. Customers can add textual content results to particular phrases or phrases as an alternative of the complete phrase. Texts may also be custom-made additional with formatting choices like daring, underline, italics, and strikethrough. Lastly, customers will be capable to schedule messages.

iPhone 14 and later fashions can have a brand new Messages through Satellite tv for pc characteristic, which permits customers to ship messages through satellite tv for pc once they haven’t got Wi-Fi or mobile service. 

The Mail app will mechanically categorize emails, a characteristic that shall be obtainable later this yr. 

The Pockets app now permits customers to faucet telephones collectively to alternate Apple Money with out requiring them to share private info like telephone numbers. 

The Journal app will now present extra statistics and insights, together with what number of entries you have had this yr, what number of days you journaled, and extra. 

There’s a new Sport Mode for iPhone, meant to assist players optimize their gaming expertise. This contains minimizing background exercise and utilizing extra responsive equipment, similar to controllers. 

The Pictures app bought what Apple dubbed its “largest ever redesign,” that includes a cleaner design, a brand new carousel with highlights that replace every day, the power to pin collections, and an improved search. 

The Messages app now helps Wealthy Communication Providers (RCS).

The Safari app was upgraded to incorporate key details about a webpage. You’ll be able to learn extra concerning the Safari upgrades beneath the MacOS part of this text. 

The Calendar app can now pull from the Reminders app for a extra seamless schedule overview. 

The brand new coaching load permits customers to achieve insights into how their exercises’ depth impacts their long-term efficiency. 

The new Vitals app will give customers a fast have a look at their most essential well being metrics, together with coronary heart charge, respiratory charge, wrist temperature, sleep period, and blood oxygen. It’s going to additionally present context to assist them make extra knowledgeable choices. If one thing appears out of the atypical, customers will obtain pings alerting them of the anomaly. 

When a consumer logs a being pregnant within the Well being App on iPhone or iPad, the Cycle Monitoring app on Apple Watch is upgraded to point out gestational age, and permits customers to log signs skilled throughout being pregnant. Pregnant customers also can ask to be reminded to take a psychological well being evaluation each month. Utilizing the Strolling Steadiness characteristic, customers may also be alerted of elevated fall danger. 

Customers can even expertise extra customizable Exercise Rings, which permit them to pause their rings once they need to take a break day with out impacting their award streaks. 

Apple Health+ was upgraded to incorporate customized For You tabs, Discover and Library tabs, search options, and enhanced awards.  

Good Stack can also be getting extra clever: it is now capable of recommend widgets when wanted mechanically, amongst different enhancements. 

With watchOS, customers will get strategies on the perfect picture choices for his or her watch face from their picture library primarily based on standards similar to aesthetics, composition, facial expressions, and extra. The face may also be custom-made with totally different sizes, layouts, and fonts. 

Examine In is obtainable on Apple Watch, even throughout exercises, to assist customers keep protected. Translate can also be coming to Apple Watch so customers can translate textual content proper from their wrists. 

One other notable change is that the double faucet gesture can be utilized to scroll by means of apps. 

The replace will characteristic a redesigned tab bar that floats above app content material. Customers can customise it to showcase their favourite apps and entry a very powerful sections of an app. You too can long-tap the bar to maneuver it round. The tab bar additionally morphs into the sidebar for added insights. 

Shareplay will enable customers to remotely management another person’s iPad or iPhone and share drawings on their screens.

In a long-awaited launch, iPads will now have a calculator app for the primary time, full with the identical interface because the one at the moment discovered on iPhones. Plus, customers can use it with the Apple Pencil by means of a brand new Math Notes expertise, which permits customers to put in writing expressions that the calculator app will remedy for them as soon as they sort the equal signal, in their very own handwriting, 

Handwriting in Notes additionally bought an improve with Good Script, which refines customers’ writing to make it extra legible whereas preserving the authenticity of the consumer’s handwriting fashion. The characteristic also can match copied and pasted textual content to the consumer’s handwriting. Typed textual content was additionally enhanced, with 5 new spotlight colours and the power to toggle sections beneath headings or subheadings. 

iPadOS 18 helps screen-sharing through SharePlay and the identical Management Middle customizations, Pictures app upgrades, Safari updates, and emoji Tapbacks present in iOS 18.

Apple unveiled MacOS Sequoia, which can embrace most of the new options that have been added to iOS 18 and iPadOS 18, together with the up to date Safari, Pictures, Messages, and the brand new Passwords administration app.

The brand new iPhone mirroring functionality on Mac permits customers to expertise their telephone nearly completely from their Mac. For instance, iPhone notifications will now be obtainable on Mac, permitting customers to work together with them and open corresponding apps, although the iPhone itself will seem locked. 

Window tiling was made doable to assist customers keep extra organized. When customers drag a window to the sting of the display screen, macOS mechanically suggests a tile place. 

Video conferences are additionally getting an improve, with new backgrounds and a preview expertise that permits you to see what you’re about to share earlier than sharing it. This characteristic works with common video conferencing functions similar to FaceTime and Zoom. 

The AI summarization software will dwell in Safari to assist customers course of content material like net pages and articles extra effectively. Safari can even help customers in discovering extra useful details about a web page they’re looking when related, similar to instructions. 

Apple additionally launched a brand new Viewer expertise, which does for video what Reader does for textual content. 

Apple unveiled the primary main improve to its not too long ago launched VisionOS — VisionOS 2. 

In Imaginative and prescient OS 2, Pictures will get an improve that permits customers to remodel any 2D picture right into a Spatial Picture, with added depth from moments already of their digicam rolls.

Spatial Personas within the Pictures app lets customers view pictures collectively, making a extra shared expertise. 

VisionOS 2 additionally helps new hand movement gestures, permitting customers to entry some settings extra simply. For instance, customers can open their fingers and faucet to achieve the house display screen or flip their wrists to see the battery degree. 

Customers who mirror their MacOS to their Imaginative and prescient Professional will quickly get the next decision and larger dimension, creating an ultrawide monitor view equal to 2 4K displays facet by facet. 

The Imaginative and prescient Professional can even embrace prepare help for journey mode, making working throughout your commute simpler. 

Imaginative and prescient Professional will now characteristic a Visitor Person possibility that permits extra customers to avoid wasting their eye and hand knowledge for 30 days. 

Customers can now personalize their House View, putting apps wherever they need.

Customers will now be capable to watch movies in an Atmosphere when utilizing Safari, even on websites similar to YouTube and Netflix. 

Apple TV brings multiview to Imaginative and prescient Professional, which is very helpful when watching sports activities video games. 

Customers can solid content material from their iPhone, iPad, or Mac to the Imaginative and prescient Professional utilizing AirPlay. 

Apple can also be making the Imaginative and prescient Professional obtainable in additional international locations beginning June 28. You’ll be able to see the complete record right here. 

When customers watch an Apple TV+ present or film, the brand new InSight characteristic on tvOS will embrace extra info similar to actor names and music titles. Customers can then simply add these music titles to their Apple Music playlist. When utilizing an iPhone as a distant, the InSight info can even seem on the smartphone. 

The Improve Dialogue characteristic was upgraded to ship better vocal readability over different parts of the film or present, similar to music or background noise, on Apple TV 4K. 

Subtitles have been optimized to mechanically seem when the language doesn’t match one of many gadgets, when customers mute, or when skipping again. 

Apple added help for 21:9 formatting for viewing content material on projectors. 

There are new, enjoyable screensavers, together with Portraits, TV and flicks, and Snoopy. 

When FaceTiming on tvOS 18, customers will now have the choice of Dwell Captions for English within the US and Canada.  

tvOS can even characteristic a redesigned Apple Health+ expertise, together with new For You, Discover, and Library areas.