Racial bias in a medical algorithm favors white patients over sicker black patients

A widely used algorithm that predicts which patients will benefit from extra medical care dramatically underestimates the health needs of the sickest black patients, amplifying long-standing racial disparities in medicine, researchers have found.

The problem was caught in an algorithm sold by a leading health services company, called Optum, to guide health care decision-making for millions of people. But the same issue almost certainly exists in other tools used by other private companies, nonprofit health systems and government agencies to manage the health care of about 200 million people in the United States each year, the scientists reported in the journal Science .

Correcting the bias would more than double the number of black patients flagged as at risk of complicated medical needs within the health system the researchers studied, and they are already working with Optum on a fix. When the company replicated the analysis on a national data set of 3.7 million patients, they found that black patients who were ranked by the algorithm as equally as in need of extra care as white patients were much sicker: They collectively suffered from 48,772 additional chronic diseases.

“It’s truly inconceivable to me that anyone else’s algorithm doesn’t suffer from this,” said Sendhil Mullainathan, a professor of computation and behavioral science at the University of Chicago Booth School of Business, who oversaw the work. “I’m hopeful that this causes the entire industry to say, ‘Oh, my, we’ve got to fix this.’”

The algorithm wasn’t intentionally racist — in fact, it specifically excluded race. Instead, to identify patients who would benefit from more medical support, the algorithm used a seemingly race-blind measure: how much patients would cost the health care system in the future. But cost isn’t a race-neutral measure of health care need. Black patients incurred about $1,800 less in medical costs per year than white patients with the same number of chronic conditions; thus the algorithm scored white patients as equally at risk of future health problems as black patients who had many more diseases.

Machines increasingly make decisions that affect human life, and big organizations — particularly in health care — are trying to leverage massive data sets to improve how they operate. They utilize data that may not appear to be racist or biased, but may have been heavily influenced by longstanding social, cultural and institutional biases — such as health care costs. As computer systems determine which job candidates should be interviewed, who should receive a loan or how to triage sick people, the proprietary algorithms that power them run the risk of automating racism or other human biases. […]

Read more – www.washingtonpost.com

AI in DC

Nearly every enterprise is experimenting with artificial intelligence and deep learning.

It seems like every week there’s a new survey out detailing the ever-increasing amount of focus that IT shops of all sizes put on the technology. If it’s true that data is the new currency, then it’s artificial intelligence that mines that data for value. Your C-suite understands that, and its why they continually push to build AI and machine learning capabilities.

Nowhere is AI/ML more impactful than in the world of government and government contractors. It’s not just the usual suspects of defense and intelligence who demand these capabilities—AI/ML is fast becoming a fact-of-life across the spectrum of government agencies. If you’re a government contractor, then you’re already seeing AI/ML in an increasing number of RFP/RFQs.

AI impacts everything

I’m a storage analyst. I don’t like to think about AI. I like to think about data. I advise my clients on how storage systems and data architecture must evolve to meet the needs of emerging and disruptive technologies. These days, those technologies all seem to be some variation of containerized deployments, hybrid-cloud infrastructure and enterprise AI. There’s no question that Artificial Intelligence is the most disruptive.

High-power GPUs dominate machine learning. Depending on the problem you’re trying to solve, that may be one GPU in a data scientist’s workstation, or it may be a cluster of hundreds of GPUs. It’s also a certainty that your deployment will scale over time in ways that you can’t predict today.

That uncertainty forces you to architect your data center to support the unknown. That could mean deploying storage systems that have scalable multi-dimensional performance that can keep the GPUs fed, or simply ensuring that your data lakes are designed to reduce redundancies and serve the needs of all that data’s consumers.

These aren’t problems of implementing AI, but rather designing an infrastructure that can support it. Most of us aren’t AI experts. We manage storage, servers, software or networking. These are all things will be disrupted by AI in the data center.

The single best way to prepare for the impacts of AI in the data center is to become educated on what it is and how it’s used. The dominant force in machine learning and GPU technology for AI is NVIDIA. Thankfully, NVIDIA has a conference to help us all out.

NVIDIA’s GPU technology conference for AI

Every spring NVIDIA hosts its massive GPU Technology Conference (GTC) near its headquarters in Silicon Valley. It’s there where 6,000+ attendees gather to hear about all aspects of what NVIDIA’s GPUs can do. This ranges from graphics for gaming and visualization, to inference at the edge, to deep learning in the enterprise. It’s one of my favorite events each year (read my recap the most recent GTC here, if interested). […]

read more – copyright by www.forbes.com

What Kind of Problems Can Machine Learning Solve?

This article is the first in a series we’re calling “Opening the Black Box: How to Assess Machine Learning Models.”

Properly deploying machine learning within an organization involves considering and answering three core questions:

The use of machine learning technology is spreading across all areas of modern organizations, and its predictive capabilities suit the finance function’s forward-looking needs. Understanding how to work with machine learning models is crucial for making informed investment decisions.

Yet, for many finance professionals, successfully employing them is the equivalent of navigating the Bermuda Triangle.

Properly deploying machine learning within an organization involves considering and answering three core questions:

1. Does this project match the characteristics of a typical machine learning problem?
2. Is there a solid foundation of data and experienced analysts?
3. Is there a tangible payoff?

Does This Project Match the Characteristics of a Typical ML Problem?

Machine learning is a subset of artificial intelligence that’s focused on training computers to use algorithms for making predictions or classifications based on observed data.

Finance functions typically use “supervised” machine learning, where an analyst provides data that includes the outcomes and asks the machine to make a prediction or classification based on similar data.

With “unsupervised” machine learning, data is provided without outcomes and the machine attempts to glean them. However, given the popularity of the supervised models within finance functions, our articles will focus on such models.

To present a very simple example in which you were attempting to train a model that predicts A + B = C using supervised machine learning, you would give it a set of observations of A, B, and the outcome C.

You would then tell an algorithm to predict or classify C, given A and B. With enough observations, the algorithm will eventually become very good at predicting C. With respect to this example, the problem is well solved by humans.

But what if the question was A+B+…+F(X) = Z?

Traditionally, humans would tackle that problem by simplifying the equation — by removing factors and introducing their own subjectivity. As a result, potentially important factors and data are not considered. A machine can consider all the factors and train various algorithms to predict Z and test its results.

In short, machine learning problems typically involve predicting previously observed outcomes using past data. The technology is best suited to solve problems that require unbiased analysis of numerous quantified factors in order to generate an outcome.

Is There a Solid Foundation of Data?

Machine learning models require data. As noted earlier, the data must also include observable outcomes, or “the right answer,” for machine learning to predict or classify.

For instance, if you are trying to predict what credit rating a private company might attain based on its financial statements, you need data that contains other companies’ financial statements and credit ratings. The ML model will look at all the financial statement data and the observable outcomes (in this case the other companies’ credit ratings), and then predict what the private company credit rating might be. […]

read more – copyright by www.cfo.com

How Machine Learning Could Impact the Future of Renewable Energy

More and more cities are looking to go green. And renewable energy is, if current trends hold, the future of the energy industry.

But as renewable energy technologies like wind farms are implemented at larger scales than ever, local officials are running into their limitations. The energy production of wind farms is hard to predict, and this makes energy grid design difficult.

Experts hope that machine learning can be applied to renewable energy to solve this problem. If it works, this new tech may make energy officials more enthusiastic about implementing renewables.

One downside of renewables is how hard it can be to predict the energy they produce. Wind speeds can vary widely from hour to hour and from day to day. You can average out how much wind a certain place gets over the course of a long period of time. And you can also use that information to figure out how much energy a wind farm may produce per year. But it’s much harder to accurately predict the energy a wind farm will produce on a given day or at a certain time.

Inaccurate predictions mean it’s harder to know if construction costs will be worth it. With renewables, too much and too little are both big problems. Create too little power and you’ll need to have supplemental energy sources at the ready. Generate too much power and you’ll need to either store that energy or waste it. And battery technology is just too expensive right now to store renewable energy at any sort of useful scale.

Machine learning technology — computer programs that use data sets to “learn” how to see patterns in information like wind speed and energy output — may be the answer to wind farms’ prediction problem.

The same machine learning tech, experts think, could be used to make green energy more predictable. In February 2019, Google announced that it was using DeepMind, the company’s in-house machine learning technology, to predict the energy output of wind farms.

The machine learning technology has already made wind farm predictions 20 percent more valuable, according to Google. And better value means that wind farms may be seen as a safer investment by municipal officials who control which kinds of energy projects get built.

Will machine learning build better wind farms? It’s hard to say. But machine learning has been successful in related fields.

The weather is notoriously difficult to predict, for many of the same reasons that it’s hard to predict wind speeds. A good prediction needs to take into account more variables than a person can keep track of — like changing levels of humidity, pressure and temperature. Predicting the weather is so hard, in fact, that IBM acquired The Weather Company to see if machine learning could make weather predictions better. The results? According to IBM, they achieved a nearly 200 percent increase in the accuracy of forecasts. […]

Read more – www.altenergymag.com

How Effective Is Artificial Intelligence In Healthcare?

Artificial intelligence (AI) and predictive modeling based on Big Data are going to be the key buzzwords in the coming decade. From banking to retail and from education to healthcare, data-intensive sectors are getting ready to be served by AI-driven computers programmed to make decisions with minimum human intervention. While interaction with “May-I-Help-You” chatbots is common on all websites today, how comfortable are we when it comes to sharing our medical history with a chatbot on visiting a hospital website? Usage of artificial intelligence in the healthcare sector is still at its infancy in India. Let’s take a look at how AI is changing healthcare globally and will do so in the future.

Can artificial intelligence change the face of healthcare globally?

Data Management

The first step in healthcare involves a compilation of previous health history and medical records. This is efficiently carried out by digital automation as all minute parameters affecting the patient’s health, and the dose-response details of all previous medication can be retracted and analyzed faster and more consistently. While consulting an individual patient with a certain set of symptoms, the doctor can pull out hundreds of other cases with similar symptoms from the database and discuss the reactions to a certain set of medications. It involves an informed discussion between both sides and brings medication to a personal care level and allows confidence building.

Performing Repetitive Analysis

Analysis of routine X-rays, ECGs, CT scans by automated robots will result in saving of a huge amount of hospital work time, and human intervention will be required in supervising only the most critical of cases. This will help hospitals in better resource management and deputation of expert manpower to saving lives in the Intensive Care Units (ICUs)and Intensive Therapy Units (ITUs).

Diagnosis And Predictive Consultation

AI can chart out a future course of action for present ailments and help outpatients and medical practitioners with predictive analysis. This can save on repetitive hospital visits for recurring ailments and provide medicare when the doctor is unavailable. Apps like Babylon in the U.K., offer medical consultation based upon the patient’s personal medical history ranked against a vast database of illness and common medical knowledge.

Digital Nurses

Boston Children’s Hospital, in 2016, developed an app for Amazon’s Alexa, which gave basic health information to parents of ill children. It also answers questions about medication and suggests a visit to a doctor after scanning symptoms. Molly, a digital nurse, has been developed by startup Sense.ly which monitors patient condition and follow-ups in between doctor visits for patients with a critical illness. In many cases, this has been found to reduce hospitalization time for patients

Creation of Drugs

Development of drugs through clinical trials takes over than a decade and involves huge costs. AI can aid in this process by scanning existing medicines including their differences in composition and effectiveness, can suggest redesigning of chemical formulations and combinations for tackling sudden medical exigencies or deadly outbreaks caused by new strains of viruses. This was found effective during the recent fight against Ebola, where AI suggested medication was found to be effective.

Health Management Apps

Wearable health trackers like FitBit, Apple and Gramin monitor real-time heart rates and activity, and charts out activity routines for the day, along with sending warning messages in case of certain parameters deteriorating, based upon the habits and needs of patients.

Copyright https://www.youthkiawaaz.com

IIoT Automation- When’s the Right Time to Invest in Automation?

6 Steps to Bring Clarity to Industrial IoT (IIoT) Automation

The allure of process automation is growing as it becomes more attainable and affordable. In the past, owning a sleek robot to assemble widgets at high speeds was as likely as having a Lamborghini sitting in your garage. But, today, automated solutions are as numerous and dependable as family sedans. Automation can provide a practical, robust and long-term solution. But, how do you know when and where to upgrade?

A good process for determining this will include the following six steps.

Begin Without Limits

1. Start with a Pie-in-the-Sky List

Make a list of ways you would automate if you had unlimited funds. The answer “everything” isn’t detailed enough. Really consider why you would automate, and what benefits the automation would bring. The “why” is especially important to the specification. Sometimes automation is viewed as the “easy” fix to problems that can and should be addressed in other ways.

Think of the process like you would when specifying your dream car. Your wish list would be long and detailed. You would pick out everything from the custom intake to the finish on the dash. Treat your automation project the same way.

2. Find the “How”

Systems integrators and automation engineers are experts in what can be accomplished reliably, using today’s technology and control systems. They can help guide you through the next step, which involves determining the actual monetary costs and benefits of automated solutions. You know what you need to automate; they know what the automation process requires.

Justify the Project; Quantify Benefits

3. Prove It

The third step is to take the most important pieces of your dream IIoT automation plan and prove or disprove the long-term financial benefits. Make sure that you can justify the costs of each automation need. For example, does the speed and production volume of your packaging line justify an automated packing station? Would manual packing operations be more financially suitable and better matched to your production needs?

Another consideration is the complexity of what you’re trying to automate. A large percentage of the automation costs is based on the number of tasks the system must perform. Conveying product and cases, for example, is one task that’s easy and cheap to automate. Assembling and printing a multi-part package where the machine must measure, rotate, count, index and assemble is going to be complex (and much more expensive to automate).

Machines aren’t human beings and must be programmed and designed. When raw material variability, multi-axis coordination and placement are involved, the machine must consistently handle these tasks with a high degree of reliability. How many moving parts are involved? How quickly must the task be performed? How many different packages or stock-keeping unit (SKU) numbers are the machine required to run? The answers to these questions can drive up the cost of automation.

When proving the value, use actual numbers. Try to assign a monetary amount to how much you can save versus how much you will spend to automate. What’s my return on investment (ROI) and does it meet my company’s requirements? A big cost factor will be the speed of the line or process—does the increase in the output (or products you can sell) justify the automation expense?

Factor in as many differences as you can determine: the cost of interruptions to existing operations for automated upgrades, manual labor expenses, differences in upkeep costs, spare part costs, operator/maintenance training costs and scrap costs, for example. Use a long-term view when trying to measure these costs, and determine your automation “break-even” point. How long will your automated line have to run to pay for the automation upgrade?

The allure of process automation is growing as it becomes more attainable and affordable. Knowing how, when and where to upgrade are essential steps to implementing industrial automation. || #IoTForAll #IoT #IIoTCLICK TO TWEET

Practical Example

Always look for the greatest impact at the lowest cost when prioritizing when and where to automate. Determine the functions to automate, how difficult automation would be and the costs versus the benefits. An example set of considerations for a packaging line is below:

You can prioritize automation tasks based on this chart. A few observations you might make:

• There are several low cost-to-automate functions that could quickly improve line speed. Case erection and case sealing are good examples. These options might be good first stage improvements.
• Consider where in the line these opportunities exist. Palletizing is an end-of-the-line function. It won’t have much effect on product output on a given day. Do you need to automate it?
• Some of the more complicated tasks might not be worth automating yet. Bundle packing is a good example. Bundle packing involves bringing multiple products together, orienting the product, and shrinking a sleeve around the entire arrangement. This is going to be a complicated process to automate. Conversely, do you have the space required for a hand-pack station? Are workers able to hand-pack at a pace that meets your output requirements? Bundle packing is a mid-stream operation and could become the rate-limiter for the line.
• Some things are worth automating because of the increase in line-speed they afford. For example, filling or case packing can greatly increase speed. Does the increase in the number of products justify the expense of automating these pieces?

Essential Elements

4. Determine Your Backbone

For anyone looking at an automated solution, there’s an essential base level of controls that must exist to make the rest possible. What is the essential base of automation for your project? Determine the basic automated structures that must exist and consider those your backbone. An experienced integrator will determine these needs and provide room for future growth at a reasonable cost.

Many automation projects move forward in stages. The backbone automation is the first stage—and maybe the only automation work you do in year one. You can spread the costs of automation by establishing this base and adding on as you move forward. Assembly could stay manual while the controls backbone is installed and the conveying is automated. Develop a timeline for the major pieces, planning future upgrades.

5. Safety and Compliance Matters

Automation is a great way to improve overall safety and compliance and generally brings a greater measure of reliability than human beings. Lockout/Tagout (LOTO) systems, machine guards, light curtains and other safety measures can be easily added. Safety practices and compliance measures must be a part of your plan.

Depending on the safety measures you will be using, the financial implications of safety systems vary. This is an area where an automation consultant can help. Automation constantly changes, and your process might change your required safety standards.

Enzyme use in consumer products manufacturing is an example of how safety can impact an automation project. Enzymes used in manufacturing can become a problem when used in large amounts. Increasing the volume of your production with automation might mean that you must account for these elevated levels of enzymes. Is an HVAC system necessary? Do workers need to wear PPE? Are room modifications required?

Will automation make your process inherently more dangerous? Are there points on the line where automation isn’t safe? Strategically choosing to require an operator to manually shut off a valve is an example. This feature ensures that the line operator must physically check the production line in specified intervals. This is an additional check for safety.

Sometimes compliance is a reason to automate. Your industry may suddenly require companies to ensure the absence of metal or foreign objects present in products. Costs of compliance would be an operational cost, as you must comply to stay in business. In many cases, automation can often provide superior reliability and speed over manual solutions.

Think You’re Finished?

6. Reduce, Review and Revise

Go back over your entire plan. Automation isn’t a stand-alone element. Usually, there are civil upgrades, logistical requirements, ergonomic considerations, equipment purchases and labor costs associated with it.

Are there areas outside of IIoT Automation where costs can be reduced? For example, are stainless tanks required, or will plastic tanks meet your needs? Do your civil and mechanical upgrades match your automation stages in terms of scope and timing? Consider the whole project, not just the automation part. Reducing your civil and mechanical scope may allow more automation up front.

Going through this entire thought process will help you develop a comprehensive automation strategy that considers costs, timeline and benefits. Much like car shopping, first, separate the wants from the needs and take a practical look at what’s possible. Once you decide to automate, it is a long-term commitment that will positively impact your business for years to come. Time invested up front will result in a much better automation solution for you and peace-of-mind moving forward.

Copyright https://www.iotforall.com

Mathematics for AI :All the important maths topics for AI and ML you need to know.

“The key to artificial intelligence has always been the representation.” — Jeff Hawkins

As we know Artificial Intelligence has gained importance in the last decade with a lot depending on the development and integration of AI in our daily lives. The progress that AI has already made is astounding with the self-driving cars, medical diagnosis and even betting humans at strategy games like Go and Chess.

The future for AI is extremely promising and it isn’t far from when we have our own robotic companions. This has pushed a lot of developers to start writing codes and start developing for AI and ML programs. However, learning to write algorithms for AI and ML isn’t easy and requires extensive programming and mathematical knowledge.

Mathematics plays an important role as it builds the foundation for programming for these two streams. And in this article, I’ve covered exactly that. Idesigned a complete articleto help you master the mathematical foundation required for writing programs and algorithms for AI and ML.

So I will directly go to the main objective of this article:

My recommendation of learning mathematics for AI goes like this:

Linear Algebra:

Linear algebra is used in Machine Learning to describe the parameters and structure of different machine learning algorithms. This makes linear algebra a necessity to understand how neural networks are put together and how they are operating.

It cover topics like this:

• Scalars, Vectors, Matrices, Tensors
• Matrix Norms
• Special Matrices and Vectors Eigenvalues and Eigenvectors
• Principle component analysis
• Singular value decomposition

Calculus:

This is used to supplement the learning part of machine learning. It is what is used to learn from examples, update the parameters of different models and improve the performance.

It cover topics like this:

• Derivatives(Scalar Derivative-Chain rule),Partial and Directional Derivative.
• Integrals
• Gradients
• Differential Operators
• Convex Optimization
• Gradient algorithms- local/global maxima and minima,SGD,NAG,MAG,Adams

Probability Theory:

The theories are used to make assumptions about the underlying data when we are designing these deep learning or AI algorithms. It is important for us to understand the key probability distributions.

It covers topics such as:

• Elements of Probability
• Random Variables
• Distributions( binomial, bernoulli, poisson, exponential, gaussian)
• Variance and Expectation
• Bayes’ Theorem, MAP, MLE
• Special Random Variables

Others

• Markov Chain
• Information Theory

From where you can learn:

• Youtube Videos
• Textbooks
• Online Course
• Google Search

Reading above topics, you will not have not only the knowledge to build your own algorithms, but also the confidence to actually start putting your algorithms to use in your next projects and learn exacly how to use concepts in real life.

Copyright https://medium.com

What Is Machine Learning?

Machine learning is one of the quickest growing technological fields, but despite how often the words “machine learning” are tossed around, it can be difficult to understand what machine learning is, precisely.

Machine learning doesn’t refer to just one thing, it’s an umbrella term that can be applied to many different concepts and techniques. Understanding machine learning means being familiar with different forms of model analysis, variables, and algorithms. Let’s take a close look at machine learning to better understand what it encompasses.

What Is Machine Learning?

While the term machine learning can be applied to many different things, in general, the term refers to enabling a computer to carry out tasks without receiving explicit line-by-line instructions to do so. A machine learning specialist doesn’t have to write out all the steps necessary to solve the problem because the computer is capable of “learning” by analyzing patterns within the data and generalizing these patterns to new data.

Machine learning systems have three basic parts:

• Inputs
• Algorithms
• Outputs

The inputs are the data that is fed into the machine learning system, and the input data can be divided into labels and features. Features are the relevant variables, the variables that will be analyzed to learn patterns and draw conclusions. Meanwhile, the labels are classes/descriptions given to the individual instances of the data.

Features and labels can be used in two different types of machine learning problems: supervised learning and unsupervised learning.

Unsupervised vs. Supervised Learning

In supervised learning, the input data is accompanied by a ground truth. Supervised learning problems have the correct output values as part of the dataset, so the expected classes are known in advance. This makes it possible for the data scientist to check the performance of the algorithm by testing the data on a test dataset and seeing what percentage of items were correctly classified.

In contrast, unsupervised learning problems do not have ground truth labels attached to them. A machine learning algorithm trained to carry out unsupervised learning tasks must be able to infer the relevant patterns in the data for itself.

Supervised learning algorithms are typically used for classification problems, where one has a large dataset filled with instances that must be sorted into one of many different classes. Another type of supervised learning is a regression task, where the value output by the algorithm is continuous in nature instead of categorical.

Meanwhile, unsupervised learning algorithms are used for tasks like density estimation, clustering, and representation learning. These three tasks need the machine learning model to infer the structure of the data, there are no predefined classes given to the model.

Let’s take a brief look at some of the most common algorithms used in both unsupervised learning and supervised learning.

Supervised Learning

Common supervised learning algorithms include:

• Naive Bayes
• Support Vector Machines
• Logistic Regression
• Random Forests
• Artificial Neural Networks

Support Vector Machines are algorithms that divide up a dataset into different classes. Data points are grouped into clusters by drawing lines that separate the classes from one another. Points found on one side of the line will belong to one class, while the points on the other side of the line are a different class. Support Vector Machines aim to maximize the distance between the line and the points found on either side of the line, and the greater the distance the more confident the classifier is that the point belongs to one class and not another class.

Logistic Regression is an algorithm used in binary classification tasks when data points need to be classified as belonging to one of two classes. Logistic Regression works by labeling the data point either a 1 or a 0. If the perceived value of the data point is 0.49 or below, it is classified as 0, while if it is 0.5 or above it is classified as 1.

Decision Tree algorithms operate by dividing datasets up into smaller and smaller fragments. The exact criteria used to divide the data is up to the machine learning engineer, but the goal is to ultimately divide the data up into single data points, which will then be classified using a key.

A Random Forest algorithm is essentially many single Decision Tree classifiers linked together into a more powerful classifier.

The Naive Bayes Classifier calculates the probability that a given data point has occurred based on the probability of a prior event occurring. It is based on Bayes Theorem and it places the data points into classes based on their calculated probability. When implementing a Naive Bayes classifier, it is assumed that all the predictors have the same influence on the class outcome.

An Artificial Neural Network, or multi-layer perceptron, are machine learning algorithms inspired by the structure and function of the human brain. Artificial neural networks get their name from the fact that they are made out of many nodes/neurons linked together. Every neuron manipulates the data with a mathematical function. In artificial neural networks, there are input layers, hidden layers, and output layers.

The hidden layer of the neural network is where the data is actually interpreted and analyzed for patterns. In other words, it is where the algorithm learns. More neurons joined together make more complex networks capable of learning more complex patterns.

Unsupervised Learning

Unsupervised Learning algorithms include:

• K-means clustering
• Autoencoders
• Principal Component Analysis

K-means clustering is an unsupervised classification technique, and it works by separating points of data into clusters or groups based on their features. K-means clustering analyzes the features found in the data points and distinguishes patterns in them that make the data points found in a given class cluster more similar to each other than they are are to clusters containing the other data points. This is accomplished by placing possible centers for the cluster, or centroids, in a graph of the data and reassigning the position of the centroid until a position is found that minimizes the distance between the centroid and the points that belong to that centroid’s class. The researcher can specify the desired number of clusters.

Principal Component Analysis is a technique that reduces large numbers of features/variables down into a smaller feature space/fewer features. The “principal components” of the data points are selected for preservation, while the other features are squeezed down into a smaller representation. The relationship between the original data potions is preserved, but since the complexity of the data points is simpler, the data is easier to quantify and describe.

Autoencoders are versions of neural networks that can be applied to unsupervised learning tasks. Autoencoders are capable of taking unlabeled, free-form data and transforming them into data that a neural network is capable of using, basically creating their own labeled training data. The goal of an autoencoder is to convert the input data and rebuild it as accurately as possible, so it’s in the incentive of the network to determine which features are the most important and extract them.

Copyright unite.ai