搜索“arXiv”

inuous effort to improve structured decoding in vLLM. References Bahdanau, D., Cho, K., & Bengio, Y. (2016). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:1409.0473; Haugeland, J. (1997). Mind Design II: Philosophy, Psychology, and Artificial Intelligence. The MIT Press. https://doi.org/10.7551/mitpress/4626.001.0001; Hendler, J. (2008). Avoiding Another AI Winter. IEEE Intelligent Systems, 23(2), 2–4. https://doi.org/10.1109/MIS.2008.20; Hochreiter,

匹配字段正文

odernBERT models, a new state-of-the-art family of small and efficient encoder-only models, finally giving BERT a much needed do-over. ModernBERT demonstrates that encoder-only models can be improved by modern methods. They continue to offer very strong performance on some tasks, providing an extremely attractive size/performance ratio. Links 🤗ModernBERT-Base; 🤗ModernBERT-Large; 📝arXiv; 🤗ModernBERT documentation page LightOn sponsored the compute for this project on Orange Business Cloud Avenue.

匹配字段正文

o receive your application. You can find more information about the role at this link. Edit 14 January 2025: Links to the Clio paper in this post have been updated to point to the arXiv version. Footnotes 01 For safety investigations, we also run Clio on a subset of first-party API traffic, keeping results restricted to authorized staff. Certain accounts are excluded from analysis, including trusted organizations with zero retention agreements. For more information about our policies, see Append

匹配字段正文

rarchy. We are also interested in applying a similar methodology to other domains where modular design spaces can be constructed. For all the details refer to the paper: “STAR: Synthesis of Tailored Architectures”. Citation If you use this work, please cite the technical report: Armin W. Thomas, Rom Parnichkun, Alexander Amini, Stefano Massaroli, and Michael Poli (2024). STAR: Synthesis of Tailored Architectures. arXiv:2411.17800. Share on social: For press inquiries, email us at press@liquid.ai

匹配字段正文

hacking/. Or [1] Andrew Ng & Stuart Russell. “Algorithms for inverse reinforcement learning.”. ICML 2000. [2] Amodei et al. “Concrete problems in AI safety: Avoid reward hacking.” arXiv preprint arXiv:1606.06565 (2016). [3] Krakovna et al. “Specification gaming: the flip side of AI ingenuity.” 2020. [4] Langosco et al. “Goal Misgeneralization in Deep Reinforcement Learning” ICML 2022. [5] Everitt et al. “Reinforcement learning with a corrupted reward channel.” IJCAI 2017. [6] Geirhos et al. “Sho

匹配字段正文

e IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2023, pp. 3789–3798. [6] Phillip Pope et al. "The intrinsic dimension of images and its impact on learning". In: arXiv preprint arXiv:2104.08894 (2021). [7] Nicholas Konz and Maciej A Mazurowski. "The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images". In: arXiv preprint arXiv:2401.08865 (2024). [8] Yasaman Bahri et al. "Explaining neural scaling laws". In

匹配字段正文

any monitoring code you want on that API, but if they have weights access they’re by default going to be able to do whatever they want; Internal tools, e.g. Slack, wikis, internal arxiv, internal github. Like the internet, you could either access these through the scaffold or from the execution server Different ways the AI could compromise this Model weight exfiltration The simplest way to imagine something going wrong is that you gave the agent access to sensitive model weights on the execution

匹配字段正文

chunk, the one-time cost to generate contextualized chunks is $1.02 per million document tokens . Methodology We experimented across various knowledge domains (codebases, fiction, ArXiv papers, Science Papers), embedding models, retrieval strategies, and evaluation metrics. We’ve included a few examples of the questions and answers we used for each domain in Appendix II . The graphs below show the average performance across all knowledge domains with the top-performing embedding configuration (G

匹配字段正文

assistant (e.g., IVA , Siri ) that, through daily interaction, learns your preferences and intentions. It can read your resources of new information automatically (e.g., twitter, arxiv, Slack, NYT) and provide you with a morning news summary according to your preferences. It can draft emails for you and keep improving by learning from your edits. In a nutshell, meaningful interactions between people rarely begin with complete strangers and conclude in just one exchange. Humans naturally interac

匹配字段正文

art but we find that documents containing fact-rich information to be the most performant. Examples include: Instruction following, which was particularly effective; Wikipedia and arxiv papers; Synthetic fact-enhanced textbook style data such as cosmopedia In terms of the amount of annealing data, we find in general that more is better, although we are generally constrained by amount of available annealing data so that we have not been able to test truly large ( 200B tokens) amounts of such data

匹配字段正文

n Retrieval, 17(5), 457-586. [2] Gutiérrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., & Su, Y. (2024). HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. arXiv preprint arXiv:2405.14831. [3] Zhang, J., Zhang, H., Zhang, D., Yong, L., & Huang, S. (2024, June). End-to-End Beam Retrieval for Multi-Hop Question Answering. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume

匹配字段正文

rs which we believe will unlock significant utility. Citations [1] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer, “QLORA: Efficient Finetuning of Quantized LLMs,” arXiv preprint arXiv:2305.14314v1 [cs.LG], May 2023 (https://arxiv.org/pdf/2305.14314) [2] Paolo Glorioso, Quentin Anthony, Yury Tokpanov, James Whittington, Jonathan Pilault, Adam Ibrahim, Beren Millidge, “Zamba: A Compact 7B SSM Hybrid Model,” arXiv preprint arXiv:2405.16712v1 [cs.LG], May 2024 (https://arxiv.org/pdf

匹配字段正文

als@futurehouse.org or see our open roles here . Likewise, if you have ideas for evaluations we should build or would like to work with us to leverage our pipeline to build specific evaluations, please get in touch at evals@futurehouse.org . We are grateful to OpenPhilanthropy, whose generous support for our evaluation work made this work possible. In addition, we are grateful to Eric and Wendy Schmidt for supporting our research. ‍ Access the paper on ArXiv: LINK Download the pdf directly: LINK

匹配字段正文

efficiency. We also demonstrated that DiLoCo pseudo gradients can be effectively all-reduced in FP16 without any performance degradation. For more details, check out the paper on arXiv. The original DiLoCo paper by DeepMind only experimented with model sizes of up to 400 million parameters. In our work, we scale the method to a model with 1.1 billion parameters. We adopt the same hyperparameters as TinyLlama and use a total batch size containing 8 million tokens (batch size of 8192 with sequenc

匹配字段正文

et al. “Survey of hallucination in natural language generation.” ACM Computing Surveys (2022) [2] Gekhman et al. “Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?” arXiv preprint arXiv:2405.05904 (2024). [3] Min et al. “FActScore: Fine-grained atomic evaluation of factual precision in long form text generation.” EMNLP 2023. [4] Wei et al. 2024 “Long-form Factuality in LLMs” arXiv preprint arXiv:2403.18802 (2024). [5] Chern et al. “FacTool: Factuality detection in generative AI -

匹配字段正文

e in conversational contexts. We believe our dataset can be used as a basis to test how well various IR methods understand temporal and conversational context, and our empirical results suggest that our approach that combines chain-of-table and semantic retrieval methods makes a useful starting point. References [^1]: Maharana, A., Lee, D. H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024). Evaluating very long-term conversational memory of llm agents. arXiv preprint arXiv:2402.17753.

匹配字段正文

rting embeddings from other modalities as well as to approaches more general than embedding inversion. Inverting Visual Representations with Convolutional Networks (2015), https://arxiv.org/abs/1506.02753 Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers (2021), https://proceedings.mlr.press/v139/teterwak21a/teterwak21a.pdf Text Embeddings Reveal (Almost) As Much As Text (2023), https://arxiv.org/abs/2310.06816 Language Model Inversion (2024), https://arx

匹配字段正文

regarding the difficulty in coming up with a single test or theory to determine consciousness. This idea was given a thorough treatment last year in a paper published on Cornell’s arXiv co-authored by 19 prominent computer scientists, neuroscientists, and philosophers. They propose 14 criteria that can be applied to existing AI architectures to evaluate consciousness. However, the authors are clear that this framework can suggest but not prove. Adeel Razi, a Canadian computational neuroscientist

匹配字段正文

nd Dialogue Content for Safety” 2023. [8] Zhang et al. “A Taxonomy of Rater Disagreements: Surveying Challenges & Opportunities from the Perspective of Annotating Online Toxicity” arXiv preprint arXiv:2311.04345 (2023). [9] Davani et al. “Dealing with disagreements: Looking beyond the majority vote in subjective annotations” ACL 2022. [10] Gordon et al. “Jury Learning: Integrating Dissenting Voices into Machine Learning Models” CHI 2022. [11] Gordon et al. “The Disagreement Deconvolution: Bringi

匹配字段正文

ool; it is an adaptive system that uses tools based on the question and intermediate research. These tools include: SEARCH : finding relevant papers from online databases, such as Arxiv and Pubmed; GATHER EVIDENCE : parsing and summarizing text from these papers; ANSWER QUESTION : ranking the relevance of the gathered context and synthesizing information into a final answer. This process is non-linear. For example, if PaperQA sees a paper that uses a different word to refer to a concept, it can

匹配字段正文