Overhaul: Swift 3, SPM and Linux

[collinhundley]

To those following this repository:

We are in the process of performing a complete rewrite of Swift AI, which will include an update to Swift 3, full Linux support, many new features, and major improvements and optimizations across the board!

Here's an overview of what's happening:

Swift 3 / Package Manager

• Create a GitHub organization account
• Use SPM modules for each component
• Update full library to Swift 3
• Break out iOS and macOS examples into separate, dedicated packages

Features

• Brand-new feed-forward neural network
• Recurrent neural network
• Convolutional neural network
• Deep learning support
• Tools for building genetic algorithms
• Signal processing APIs (Fourier transforms, etc)

Linux

• Use a C BLAS library for optimized vector calculations
• OpenCL/CUDA acceleration

iOS / macOS

• Metal acceleration
• Experiment with Apple's new neural network tools for certain operations (iOS 10+)

API

• More intuitive API for creating, training and updating neural networks
• More intelligent training methods
• Potentially support Caffe models

Optimizations

• Better memory management. The neural network training routines are currently optimized for speed, at the expense of memory (large up-front allocations are made in order to avoid small heap allocations later). We can achieve a better balance.
• Make use of UnsafePointer when appropriate. Bypassing Swift's type safety in certain places can give us a big performance boost.
• Better random generation of weights. Some applications need weights generated along a normal distribution instead of a uniform distribution.
• Better support for alternative activation functions

Website

• A place for viewing documentation, sharing code and getting help
• This would be fun to have. But it also costs $$

Help and Contributions

Since I'll be making some dramatic changes to this library, I probably won't be accepting contributions until I've at least settled on the new APIs and standards. However, there are a few places where I could use some help:

• Feature/API suggestions: I'm especially interested in feedback from people who have used other machine learning frameworks like TensorFlow, Torch, etc.
• Deep learning experts: If this is you, let's be in touch.
• Logo and branding: The Swift bird is cool, but apparently some fruit company owns it.
• Sponsorship: Asking for money is lame. But it helps things move faster.

Thanks to everybody for using Swift AI. I'm excited to see what we build!

Collin

[romaHerman]

Hi, Collin

Thanks for the update !
I hope I will be able to contribute to this project soon

[mdab121]

Can't wait to contribute as well!

[Jxrgxn]

Thanks Colin.

[popaaaandrei]

Hello Colin,

Thank you for your efforts.
Any plans on supporting NVIDIA Accelerated libs CUDA, cuDNN, cuBLAS, etc. ?

Thank you!

[collinhundley]

@popaaaandrei, GPU learning on Linux is a high priority for me. Most likely this will mean CUDA, although OpenCL is an option too. If you - or anybody you know - is an expert with those libraries, we'll have to be in touch when the time comes.

[popaaaandrei]

@collinhundley
This might be the missing link: https://github.com/rxwei/cuda-swift

[simonnarang]

Maybe we can use gitter.im for now?

[collinhundley]

@simonnarang actually, I just created a Slack channel :)

swift-ai.slack.com

[NikoXu]

A noble quest! I can't wait for the time coming. But I am a ML beginner. Wish I could grow with the new project.
Thanks Colin.

[mogarg]

Hey Collin, Please let me know how can I join the slack channel. Thanks, Mohit Garg

…

On Thu, Dec 15, 2016 at 8:43 PM, Collin Hundley ***@***.***> wrote: @simonnarang <https://github.com/simonnarang> actually, I just created a Slack channel :) swift-ai.slack.com — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#60 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALy_fJIo1nmpsSbps9khQSW2Pk-9S_Opks5rIezNgaJpZM4KIAbc> .

[simonnarang]

@collinhundley 👍 Sweet! To make it easy for people to join, maybe https://github.com/rauchg/slackin would work?

[collinhundley]

@mogarg @simonnarang we'll use Slackin as soon as we have a site running. It'll take some time, but hold tight! Once that's ready I'll open the Slack channel up to the public.

[lzackx]

WoW~ Can't wait to contribute x 1024

[tianyouhui]

exciting...

[williamxiewz]

Wow

[ZENTRALALEX]

I can`t wait to see what you will come up with :) Can you imagine when we can have a look at parts of the new project? Also great to see that you want to support Linux. It would be great to be able to use it also in a server project.

[DevAndArtist]

Any progress on this?

[iamjono]

Hey if you want to use a pure Swift version of Slackin, check out https://github.com/PerfectServers/SwiftSlack :)

[collinhundley]

Hey everyone! Just an update to this thread: I've pushed a new release which brings Swift 3.1 support and a plethora of other requested changes. This is just part 1 - more great updates coming soon!

See the release for a semi-complete list of changes.

[adamduracz]

Any news on Linux support? My research project is looking around for ML libraries to do training and inference on an embedded board, i.e. using (ARMV8 multi-core) CPU rather than a GPU, and it seems like yours is the closest thing available.

[JoeriBultheel]

Hi,

Very much interested in a on-device-learning RNN implementation. Is there any progress being made regarding that? Would love to contribute / stay informed.

Thnx!