HN
login
Ask HN: How do looped transformers work with respect to context windows?

Posting here as I haven't found the answer anywhere. I know that GPT Astra uses looping instead of reasoning tokens. My specific question is this: reasoning tokens pollute the context and makes compaction kick in…

1 points•simianwords•about 4 hours ago•0 comments•
#looped#context#transformers#looped transformers#windows#respect#context windows#transformers work
Posting here as I haven't found the answer anywhere.

I know that GPT Astra uses looping instead of reasoning tokens. My specific question is this: reasoning tokens pollute the context and makes compaction kick in quickly. Does this also happen to looped transformers like Astra?

0 comments

No comments yet.

Related stories