(#2utweua) @prologic Sorry had to finish my "day job" and missed. Time zones are so disruptive to communications. I haven't used IRC in a few decades, need to remember how it works and where the client is. Feeling really old now. :-|

Seem to be getting duplicates when I sync, I think I am still missing something in integrated Yarn/twtxt.net and my local text tools.

@rsdoiel @prologic (#2utweua) Maybe a bit offtopic, but: If I pull twtxt.net's feed, change its URL and sync the changed file back into twtxt.net, will the feed in the pod retain the new URL? Even if then I keep editing the feed by posting into it using the pod?

(#2utweua) @rsdoiel Don't worry, I've used IRC for over 25years and counting now 😅 Its still the best open communications medium around really 🤣 Just head over to Yarn.social#collaboration and follow the links. I _tried_ to make it easy by linking the Libera Web Chat client 👌

(#2utweua) @rsdoiel Hmmm I think you might have run into a bug 🤣 I might need your help figuring this out 👌

(#2utweua) @rsdoiel Hmmm 🤔

(#2utweua) @rsdoiel Where is your Github feed you're mirroring? 🤔 The only reason for "dupes" is the way hashes are calculated. They are based on what's called a "Hashing URI", essentially the first `# url = ` field in your feed's metadata. I _think_ (but would have to confirm) that if you're trying to sync your Github hosted `twtxt.txt` feed to your twtxt.net feed (on my pod), if you have a `# url =` field in your Github side that isn't the same value as your feed on your twtxt.net feed, then you'll end up with dupes. You _can_ have multiple `# url =` fields, but the first one is used as the "Hashing URI" and if you're syncing, it **must** be the same on both sides.

(#2utweua) Does this make sense? 🤔 In fact I _probably_ wouldn't have any Metadata on your Github feed side at all, just delete it and re-sync (after fixing the dupes), and then help me add some additional features to add additional KV pairs to your feed's Metadata in `yarnd` (the backend for Yarn.social) 🙏

(#2utweua) Hmmm, I not understanding something. I'm pretty sure this is pilot error on my part. 1. I changed the metadata (url =) in my github.com copy twtxt.txt to match the and ran `yarnc sync twtxt.txt` locally. Still propagating duplicates (most duplicates have no hash) 2. Replaced my local twtxt.txt file with contents of cleaned up duplicates, commit change to Github, ran `yarnc sync twtxt.txt`. Propogating duplicates still. In the web UI of twtxt.net I don't see a "delete" button to remove duplicates. In the curled twtxt.txt file there is odd formatting. The text would wrap to a new line and occasionally this would run into one of the duplicate post's timestamps (i.e. no newline between previous post and next post). Garbage in garbage out maybe?

(#2utweua) @prologic I think you're making sense but my brain is dim 🙂