One criticism of OpenAI is related to the challenge of evaluating and providing feedback for their language models. The models are constantly updated, making it difficult to compare different versions and track improvements .
Another criticism is that OpenAI has a closed policy when it comes to sharing their models and code. This has led to calls for more open sourcing and greater engagement with the open source community .
There are also concerns about the commoditization of large language models, with some suggesting that there may be a race to the bottom in terms of pricing and quality .
In addition, there are discussions around the need for better benchmarking of language models, especially in terms of real-world use cases and practical considerations like latency, cost, and throughput .
It's important to note that these criticisms are not unique to OpenAI and are part of wider debates and challenges in the field of AI and natural language processing .