[ANN] Link - The Blockchain File Sharing Protocol
-
I’ll be honest. I’m salivating over what I think is a good idea. I wish I had something to contribute to this idea, but I am not technical.
-
[quote name=“mnstrcck” post=“40862” timestamp=“1386383755”]
Legally, having the entire blockchain downloaded can result in issues with content contained within. Security and anonymity should be a key concern. How would this be solved?
[/quote]Speaking as someone who instead would want it to be used as a disruption tool, security and anonymity is super important but at the same time I do want a technology that is so powerful and persuasive enough that governments simply are forced to give up trying to shape laws to deal with it.Force all content to have a unique public PGP key? People trading and using private keys will be 3rd party from the project on forums and communities not affiliated with the project.
-
[quote name=“mnstrcck” post=“40862” timestamp=“1386383755”]
KnowledgeCoin or OpenCoin or something like that might actually be a better name. It has so many uses to: [b]Links[/b] to torrent magnets, manifesto uploads, images can be shared in textual formats [base64],
[/quote]
I think “Link” is a great name, especially because it describes exactly what it is. [i][b]Linkster[/b][/i], lulz.
[quote author=mnstrcck link=topic=5558.msg40862#msg40862 date=1386383755]
Which brings me to the search function:If a “green” - “red” / like - dislike function can be added that users vote on entries in the blockchain with - which then gets added to the it as a sort of transaction we could provide for the ability to use the search function as a means of seeing what’s POPULAR in the last few blocks. And, of course, it can be used to further thwart abuse as users can choose to filter things. Limitations on “transactions” like this can be placed - or a fee instituted so voting also costs money. But it never removes anything - it’s all user specific - just sort of meta-data to show what’s popular.
[/quote]
Great idea!
Also, nodes could carry specific information perhaps or choose which to carry (like a library), which I believe (this ability) is built into the bitcoin blockchain as per a LTB discussion recently. So a node could specialize in certain links (music/ebook/data/images/video/information/etc) and only download those they subscribe to … like a newsreader on Usenet.
-
[quote name=“mnstrcck” post=“40862” timestamp=“1386383755”]
Users can create tags/keywords [like hashtags] to put things up and make things easy to search for.
[/quote]That feature will actually be built into the protocol. :)
[quote]
Which brings me to the search function:If a “green” - “red” / like - dislike function can be added that users vote on entries in the blockchain with - which then gets added to the it as a sort of transaction we could provide for the ability to use the search function as a means of seeing what’s POPULAR in the last few blocks. And, of course, it can be used to further thwart abuse as users can choose to filter things. Limitations on “transactions” like this can be placed - or a fee instituted so voting also costs money. But it never removes anything - it’s all user specific - just sort of meta-data to show what’s popular.
[/quote]That’s something I hadn’t considered adding to the protocol, but there’s no reason it couldn’t be added at a later date! I’m going to start with the basic search result + keywords, and then we can move onto other meta-data support.
[quote]
Not sure how this would work out technically.
[/quote]It’s trivial to do fortunately, albeit pretty expensive for the user.
[quote]
Legally, having the entire blockchain downloaded can result in issues with content contained within. Security and anonymity should be a key concern. How would this be solved?
[/quote]With encryption of course. That’s what Tor is for. This is already an issue with the Bitcoin blockchain as there’s already copyrighted content encoded in it, so adding this won’t change anything about that situation.
-
First let’s discuss file sharing:
We cannot have a bloated blockchain for file sharing. Filesharing blocks have to expire or be reusable in some way. The filesharing blocks should simply be references that improve finding chunks of data. There was once a filesharing system OFFSystem, a brightnet. People send random strings that contain zero information until combined with other random strings to produce the data. There is no way to say who has any data until it is combined. No one knows what you have and no one can sue.Seems like a good starting point.
To Tuck regarding exchanges:
I’d like to jump on this with a concept: FLUXchannel
We all know how difficulty works. Get a certain minimum of 0s and you get your reward. But this means 0s from the beginning of the binary representation. 000000blah works. blah000000blah does not. Pools reject hashes which have the same probability but happen to be shifted by a few bits. We should use these for other purposes. This approach beats Mastercoin and Coin Coloring. Both suffer from bloat effects.I have been thinking about my earlier exchange idea. And it occurred to me file sharing is a kind of exchange. You get data as long as you’re seeding. Up until now we have had big forums that spend all day arguing about who is leeching and who is contributing and it’s frankly sad to watch. Suppose there were some steps to converting the rejected hashes into coins and people could devote some hashes to someone else’s mining for contributing data. At the end of the work all that remains in the blockchain is the coins with little or no trace of the data or work exchange.
Mining levels: These levels represent different levels of reward
0 - Confirmation mining
a. too low = fail to receive reward.
b. high hash = receive reward1 - Exchange mining
a. rotating someone else’s failed hashes based on whether they are delivering on promises. the incentive is for contributors to be able to trade the level 1 reward based on item and value for actual FTC
b. rotating hashes the other way based on failure to deliver or extension of work. the incentive is for contributors to recover uncommitted funds.2 - Work mining
a. splitting 0 strings in hashes and rotating a portion which cannot be retracted when it completes. this would be used to pay someone in installments.
b. combining 0 strings in hashes to guarantee a minimum payment. this would be used to make sure that if a payment is transmitted for partial work or sale it cannot be recalled once it arrives.3 - Conversion mining
a. broadcasting high difficulty hashwork to other chains for trade. if you hit a high difficulty hash you can send that across chains and receive a larger reward.
b. broadcasting low difficulty hashwork to other chains for trade. if you hit a low difficulty hash you can still get paid for the work by having it approved on a different network.Each level higher requires some of the lower level interactions. It will also require cryptocoiners to agree on a value (work not price) negotiation scheme. 200 FTC at a high difficulty is the same as 50 FTC reward at low difficulty.
How does this apply to file sharing?
As I mentioned with OFFSystem you have random blocks that are put together. The number of combinations increase geometrically.With 2 random blocks you can have a 3rd block.
3, 3 extra blocks.
4, 6 extra blocks.
5, 10
6, 15Now most of these results will be random as well, but with the rotations and such they can be converted into useful chunks of data. I would limit the coin work to rotations, but with filesharing we can add other operations to the rotations:
ROT - straight forward rotation
XOR - simple way to combine blocks to create other blocks
ADD - add the values with overflow remainder
REV - reverse subsection of bitsOne thing that comes to mind is common data, like schema and file format data will take up much less space.
-
There is something else that happens in the above scheme:
It allows low hashrate miners to get money back for someone who is getting ripped off. It’s easier for people to share hashes than it is to send money around. This would create a market where someone could not hold out in trading for content or work. It also turns hashes into something that may contain the information people are looking for even if no one can find missing pieces. -
[quote name=“zerodrama” post=“40876” timestamp=“1386385132”]
First let’s discuss file sharing:
We cannot have a bloated blockchain for file sharing. Filesharing blocks have to expire or be reusable in some way. The filesharing blocks should simply be references that improve finding chunks of data. There was once a filesharing system OFFSystem, a brightnet. People send random strings that contain zero information until combined with other random strings to produce the data. There is no way to say who has any data until it is combined. No one knows what you have and no one can sue.Seems like a good starting point.
[/quote]Yes it does. I’d never heard of a brightnet, but after reading shortly it sounds like a genius idea.
[quote author=zerodrama link=topic=5558.msg40876#msg40876 date=1386385132]
To Tuck regarding exchanges:
I’d like to jump on this with a concept: FLUXchannel
We all know how difficulty works. Get a certain minimum of 0s and you get your reward. But this means 0s from the beginning of the binary representation. 000000blah works. blah000000blah does not. Pools reject hashes which have the same probability but happen to be shifted by a few bits. We should use these for other purposes. This approach beats Mastercoin and Coin Coloring. Both suffer from bloat effects.
[/quote]I’ve never peeked behind the curtains of the great Pool Wizard, but if what you’re saying is valid it sounds like an excellent way for it to be handled.
[quote author=zerodrama link=topic=5558.msg40876#msg40876 date=1386385132]
Suppose there were some steps to converting the rejected hashes into coins and people could devote some hashes to someone else’s mining for contributing data. At the end of the work all that remains in the blockchain is the coins with little or no trace of the data or work exchange.
[/quote][Mind Blown.gif] (Technical forum adjustment)
-
Is there a GPU miner for this thang yet? All I’m getting are rejects!
Any pools or is it being premined as I type?
I don’t see any source code (Warning possible wallet stealer or Scam Coin!).
Is this scrypt?
What port do I use?
Any other nodes?
-
I am very interested in seeing this idea through, it could be one of the major talking points which takes us from “clone” to Feathercoin. I think having torrents embedded in the blockchain is a great first step, if we could get a website like TPB on board it could be huge!
Kevlar: Would this protocol have to be accessed via TOR?
PS. What is in the bitcoin block-chain which is copyrighted?
-
[quote name=“Vidicus” post=“41012” timestamp=“1386406659”]
I think having torrents embedded in the blockchain is a great first step, if we could get a website like TPB on board it could be huge!
[/quote]It would require a bit of resources to do, but it’s entirely possible to do it. Getting the index into the blockchain is easy. Having the coins to spend… a bit trickier.
You wouldn’t need them to do anything. It’s the type of project a clever kid with some python skills could whip up in an hour.
[quote]
Kevlar: Would this protocol have to be accessed via TOR?
[/quote]Nope, it just layers into the existing blockchain protocol. No changes to the blockchain required.
[quote]
PS. What is in the bitcoin block-chain which is copyrighted?
[/quote]Well I know someone embedded DeCSS in there, which is the famously copyrighted number. My buddy Dan Kaminsky did a bunch of work on this a while back.
-
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project! -
[quote name=“Vidicus” post=“41021” timestamp=“1386407620”]
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project!
[/quote]I’m already hard at work on it. Your donations are GREATLY appreaciated. :)
The python would have to read the pirate bay database, and insert it into the blockchain by spending the coins using the Link protocol. I’ve not yet calculated what the cost of a message will be, but I do estimate you can do several hundred thousand torrents for 1 FTC.
-
[quote name=“Kevlar” post=“41054” timestamp=“1386410677”]
[quote author=Vidicus link=topic=5558.msg41021#msg41021 date=1386407620]
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project!
[/quote]I’m already hard at work on it. Your donations are GREATLY appreaciated. :)
The python would have to read the pirate bay database, and insert it into the blockchain by spending the coins using the Link protocol. I’ve not yet calculated what the cost of a message will be, but I do estimate you can do several hundred thousand torrents for 1 FTC.
[/quote]So are we just storing torrent files in the blockchain? Torrents are notoriously resistant to adaptation. There is no torrent modify protocol or dynamic torrent. For example, there is no torrent proxy which can turn a folder being shared into a torrent file in real time.
I’ll work on FLUX and see how it fits here.
-
[quote name=“zerodrama” post=“41185” timestamp=“1386429718”]
[quote author=Kevlar link=topic=5558.msg41054#msg41054 date=1386410677]
[quote author=Vidicus link=topic=5558.msg41021#msg41021 date=1386407620]
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project!
[/quote]I’m already hard at work on it. Your donations are GREATLY appreaciated. :)
The python would have to read the pirate bay database, and insert it into the blockchain by spending the coins using the Link protocol. I’ve not yet calculated what the cost of a message will be, but I do estimate you can do several hundred thousand torrents for 1 FTC.
[/quote]So are we just storing torrent files in the blockchain? Torrents are notoriously resistant to adaptation. There is no torrent modify protocol or dynamic torrent. For example, there is no torrent proxy which can turn a folder being shared into a torrent file in real time.
I’ll work on FLUX and see how it fits here.
[/quote]That’s one possibility and you can do that with the Link protocol but I’m working on something a little more efficient in terms of storage. Why store the whole torrent when you can store a tiny fraction and accomplish the same thing?
As for modifying torrents, torrents are like URLs: you don’t modify one, you use a new one. That’s an important feature, because it means the same data will always be downloaded for a given torrent. Want different data? You wanted a different torrent.
Making them a directory at a time is trivial.
-
Is there any way the concept of holding the magnet files can encourage or push people to use i2p for further security?
I’m just envisioning people running the Link through Tor and going “Huh! It’s good enough for that. Let me go ahead run my torrent client through it.” and having a small nightmare about it.
-
[quote name=“Kevlar” post=“41250” timestamp=“1386439601”]
That’s one possibility and you can do that with the Link protocol but I’m working on something a little more efficient in terms of storage. Why store the whole torrent when you can store a tiny fraction and accomplish the same thing?
[/quote]See that’s what I was hoping for. Because many torrents are actually collections of files. This makes more sense. I still think link could be more dynamic. Like forking a linkchain for different remixes. Or updates to documents in progress.
[quote]
As for modifying torrents, torrents are like URLs: you don’t modify one, you use a new one. That’s an important feature, because it means the same data will always be downloaded for a given torrent. Want different data? You wanted a different torrent.
[/quote]Except we also have urls which are actually queries formatted as directories for search engines to crawl, which would mean blockchain search engines.
[quote]
Making them a directory at a time is trivial.
[/quote]I really should just build this feature and stop whining about it.
-
[quote name=“HopeStillFlies” post=“41254” timestamp=“1386440056”]
Is there any way the concept of holding the magnet files can encourage or push people to use i2p for further security?I’m just envisioning people running the Link through Tor and going “Huh! It’s good enough for that. Let me go ahead run my torrent client through it.” and having a small nightmare about it.
[/quote]Well they can run the blockchain download through the Tor network. That’s easy to do. I personally run my uTorrent over Tor and while I won’t max my bandwidth that way, it works well enough.
-
Ok, I’ve made a few changes to the initial pass. Specifically I’ve changed it from a fixed-length block-based encoding scheme to a variable-length stream-based encoding scheme based on op-codes to maximize compression and minimize wasted space in the blockchain.
Now is the time to get your op-codes submitted for inclusion. The list is far from complete, but please read the description of the ones suggested so far and if you have any ideas for other ones please let me know!
-
Is there anything someone who doesn’t have cryptos to donate, and doesn’t have the underlying computer skills, can do to help? It think this idea is brilliant.
-
[quote name=“Kevlar” post=“41313” timestamp=“1386448154”]
Ok, I’ve made a few changes to the initial pass. Specifically I’ve changed it from a fixed-length block-based encoding scheme to a variable-length stream-based encoding scheme based on op-codes to maximize compression and minimize wasted space in the blockchain.Now is the time to get your op-codes submitted for inclusion. The list is far from complete, but please read the description of the ones suggested so far and if you have any ideas for other ones please let me know!
[url=https://github.com/TSavo/Link/blob/master/README.md]https://github.com/TSavo/Link/blob/master/README.md[/url]
[/quote]I would do this for opcodes:
1st byte = module slot
2nd byte = module functionSo what you have defined so far is module 0 - the core link module. I would move 10, 11, FE, FF to 0A, 0B, 0D, 0E.
Module function 0 for all modules is the module key. Module function F for all modules is the extension range. Function 0 would identify the module we expect to be installed there. Function F would allow us to extend a module over several slots.The module key could be a digest of module name, version, VFW Post, geocaching GPS info, and other identifiers - they wouldn’t need to be listed, just the digest would need to match. The extension range would tell us the beginning and ending module slot which are allocated.
So core module/module key = Opcode 00. The extension key just points to module slot 0 as beginning and end. Say module 1 is a track list and module 2 is a list of ratings. Then track list/module key = Opcode 10 and ratings list/module key = Opcode 20. Both would also have extension range with beginning and end 1,1 and 2,2.
This means 14 functions per module slot. It allows for extreme extending of any concepts we want to add without getting lost. Core module in slot 0 is reserved for basic link protocol. I would suggest reserving slot F as well. The rest can be used by extenders. That leaves 14 slots of 14 functions each, for whatever we want to do. There is some duplication of info, but that may be the best way to do it.
If this isn’t too weird, I’ll set up FLUX (Flux Link Unit eXchange) functions within this same scheme.
One thing I would add (Peter Falk voice): some of these functions may require read write capability, but that’s for another time.
One favor Kevlar, I want to be able to read these streams from nodejs just for correctness in encoding. I might need to check with you on that.
