Cerebras has set a new record for AI inference speed, serving Llama 3.1 8B at 1,850 output tokens/s and 70B at 446 output tokens/s.

@CerebrasSystems has just launched their API inference offering, powered by their custom wafer-scale AI accelerator chips.

Llama 3.1 8B provider analysis:

---

Analysis of API for Llama 3.1 Instruct 8B across performance metrics including latency (time to first token), output speed (output tokens per second), price and others. API benchmarked include Microsoft Azure, Amazon Bedrock, Groq, Together.ai, Perplexity, Fireworks, Cerebras, Lepton AI, Deepinfra, and OctoAI.

The wing dynamics of flying animal species have been the inspiration for numerous flying robotic systems. While birds and bats typically flap their wings using the force produced by their pectoral and wing muscles, the processes underlying the wing movements of many insects remain poorly understood.

Researchers at Ecole Polytechnique Fédérale de Lausanne (EPFL, Switzerland) and Konkuk University (South Korea) recently set out to explore how herbivorous insects known as rhinoceros beetles deploy and retract their wings. The insight they gathered, outlined in a paper published in Nature, was then used to develop a new flapping microrobot that can passively deploy and retract its wings, without the need for extensive actuators.

“Insects, including beetles, are theoretically believed to use thoracic muscles to actively deploy and retract their wings at the wing bases, similarly to birds and bats,” Hoang-Vu Phan, the lead author of the paper, told Tech Xplore. “However, methods of recording or monitoring muscular activity still cannot determine which muscles beetles use to deploy and retract their wings nor explain how they do so.”

Over the past couple of decades, computer scientists have developed a wide range of deep neural networks (DNNs) designed to tackle various real-world tasks. While some of these models have proved to be highly effective, some studies found that they can be unfair, meaning that their performance may vary based on the data they were trained on and even the hardware platforms they were deployed on.

For instance, some studies showed that commercially available deep learning–based tools for facial recognition were significantly better at recognizing the features of fair-skinned individuals compared to dark-skinned individuals. These observed variations in the performance of AI, in great part due to disparities in the training data available, have inspired efforts aimed at improving the fairness of existing models.

Researchers at University of Notre Dame recently set out to investigate how hardware systems can contribute to the fairness of AI. Their paper, published in Nature Electronics, identifies ways in which emerging hardware designs, such as computing-in-memory (CiM) devices, can affect the fairness of DNNs.