GraphineOS seems to set the benchmark for secure de-googled android phones and has a very short list of supported devices. I think I’d suggest starting with one of those, and once support eventually drops, if you’re comfortable with a reduced security capability, looking to lineageOS or similar. I think if Graphine supports a phone, it’s pretty much guaranteed to have support on the more general OSs.

For a while I looked at ruggedized smartphones (some with removable batteries!) that were supported by lineageOS and others. I didn’t find one I was convinced would hold up as long as I wanted, and I had security concerns so I ended up getting a decent secondhand phone with guaranteed security support for a few years and putting it in a good case.

Sometimes I check in on various raspberry pi smartphone projects. I love the idea and think it’d probably be able to last the longest (or be turned into something else after an upgrade) but I don’t think any feel reliable enough to me yet.

That beats connecting the speaker jack to the mic jack and recording with audacity.