[ANN] Link - The Blockchain File Sharing Protocol
-
First let’s discuss file sharing:
We cannot have a bloated blockchain for file sharing. Filesharing blocks have to expire or be reusable in some way. The filesharing blocks should simply be references that improve finding chunks of data. There was once a filesharing system OFFSystem, a brightnet. People send random strings that contain zero information until combined with other random strings to produce the data. There is no way to say who has any data until it is combined. No one knows what you have and no one can sue.Seems like a good starting point.
To Tuck regarding exchanges:
I’d like to jump on this with a concept: FLUXchannel
We all know how difficulty works. Get a certain minimum of 0s and you get your reward. But this means 0s from the beginning of the binary representation. 000000blah works. blah000000blah does not. Pools reject hashes which have the same probability but happen to be shifted by a few bits. We should use these for other purposes. This approach beats Mastercoin and Coin Coloring. Both suffer from bloat effects.I have been thinking about my earlier exchange idea. And it occurred to me file sharing is a kind of exchange. You get data as long as you’re seeding. Up until now we have had big forums that spend all day arguing about who is leeching and who is contributing and it’s frankly sad to watch. Suppose there were some steps to converting the rejected hashes into coins and people could devote some hashes to someone else’s mining for contributing data. At the end of the work all that remains in the blockchain is the coins with little or no trace of the data or work exchange.
Mining levels: These levels represent different levels of reward
0 - Confirmation mining
a. too low = fail to receive reward.
b. high hash = receive reward1 - Exchange mining
a. rotating someone else’s failed hashes based on whether they are delivering on promises. the incentive is for contributors to be able to trade the level 1 reward based on item and value for actual FTC
b. rotating hashes the other way based on failure to deliver or extension of work. the incentive is for contributors to recover uncommitted funds.2 - Work mining
a. splitting 0 strings in hashes and rotating a portion which cannot be retracted when it completes. this would be used to pay someone in installments.
b. combining 0 strings in hashes to guarantee a minimum payment. this would be used to make sure that if a payment is transmitted for partial work or sale it cannot be recalled once it arrives.3 - Conversion mining
a. broadcasting high difficulty hashwork to other chains for trade. if you hit a high difficulty hash you can send that across chains and receive a larger reward.
b. broadcasting low difficulty hashwork to other chains for trade. if you hit a low difficulty hash you can still get paid for the work by having it approved on a different network.Each level higher requires some of the lower level interactions. It will also require cryptocoiners to agree on a value (work not price) negotiation scheme. 200 FTC at a high difficulty is the same as 50 FTC reward at low difficulty.
How does this apply to file sharing?
As I mentioned with OFFSystem you have random blocks that are put together. The number of combinations increase geometrically.With 2 random blocks you can have a 3rd block.
3, 3 extra blocks.
4, 6 extra blocks.
5, 10
6, 15Now most of these results will be random as well, but with the rotations and such they can be converted into useful chunks of data. I would limit the coin work to rotations, but with filesharing we can add other operations to the rotations:
ROT - straight forward rotation
XOR - simple way to combine blocks to create other blocks
ADD - add the values with overflow remainder
REV - reverse subsection of bitsOne thing that comes to mind is common data, like schema and file format data will take up much less space.
-
There is something else that happens in the above scheme:
It allows low hashrate miners to get money back for someone who is getting ripped off. It’s easier for people to share hashes than it is to send money around. This would create a market where someone could not hold out in trading for content or work. It also turns hashes into something that may contain the information people are looking for even if no one can find missing pieces. -
[quote name=“zerodrama” post=“40876” timestamp=“1386385132”]
First let’s discuss file sharing:
We cannot have a bloated blockchain for file sharing. Filesharing blocks have to expire or be reusable in some way. The filesharing blocks should simply be references that improve finding chunks of data. There was once a filesharing system OFFSystem, a brightnet. People send random strings that contain zero information until combined with other random strings to produce the data. There is no way to say who has any data until it is combined. No one knows what you have and no one can sue.Seems like a good starting point.
[/quote]Yes it does. I’d never heard of a brightnet, but after reading shortly it sounds like a genius idea.
[quote author=zerodrama link=topic=5558.msg40876#msg40876 date=1386385132]
To Tuck regarding exchanges:
I’d like to jump on this with a concept: FLUXchannel
We all know how difficulty works. Get a certain minimum of 0s and you get your reward. But this means 0s from the beginning of the binary representation. 000000blah works. blah000000blah does not. Pools reject hashes which have the same probability but happen to be shifted by a few bits. We should use these for other purposes. This approach beats Mastercoin and Coin Coloring. Both suffer from bloat effects.
[/quote]I’ve never peeked behind the curtains of the great Pool Wizard, but if what you’re saying is valid it sounds like an excellent way for it to be handled.
[quote author=zerodrama link=topic=5558.msg40876#msg40876 date=1386385132]
Suppose there were some steps to converting the rejected hashes into coins and people could devote some hashes to someone else’s mining for contributing data. At the end of the work all that remains in the blockchain is the coins with little or no trace of the data or work exchange.
[/quote][Mind Blown.gif] (Technical forum adjustment)
-
Is there a GPU miner for this thang yet? All I’m getting are rejects!
Any pools or is it being premined as I type?
I don’t see any source code (Warning possible wallet stealer or Scam Coin!).
Is this scrypt?
What port do I use?
Any other nodes?
-
I am very interested in seeing this idea through, it could be one of the major talking points which takes us from “clone” to Feathercoin. I think having torrents embedded in the blockchain is a great first step, if we could get a website like TPB on board it could be huge!
Kevlar: Would this protocol have to be accessed via TOR?
PS. What is in the bitcoin block-chain which is copyrighted?
-
[quote name=“Vidicus” post=“41012” timestamp=“1386406659”]
I think having torrents embedded in the blockchain is a great first step, if we could get a website like TPB on board it could be huge!
[/quote]It would require a bit of resources to do, but it’s entirely possible to do it. Getting the index into the blockchain is easy. Having the coins to spend… a bit trickier.
You wouldn’t need them to do anything. It’s the type of project a clever kid with some python skills could whip up in an hour.
[quote]
Kevlar: Would this protocol have to be accessed via TOR?
[/quote]Nope, it just layers into the existing blockchain protocol. No changes to the blockchain required.
[quote]
PS. What is in the bitcoin block-chain which is copyrighted?
[/quote]Well I know someone embedded DeCSS in there, which is the famously copyrighted number. My buddy Dan Kaminsky did a bunch of work on this a while back.
-
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project! -
[quote name=“Vidicus” post=“41021” timestamp=“1386407620”]
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project!
[/quote]I’m already hard at work on it. Your donations are GREATLY appreaciated. :)
The python would have to read the pirate bay database, and insert it into the blockchain by spending the coins using the Link protocol. I’ve not yet calculated what the cost of a message will be, but I do estimate you can do several hundred thousand torrents for 1 FTC.
-
[quote name=“Kevlar” post=“41054” timestamp=“1386410677”]
[quote author=Vidicus link=topic=5558.msg41021#msg41021 date=1386407620]
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project!
[/quote]I’m already hard at work on it. Your donations are GREATLY appreaciated. :)
The python would have to read the pirate bay database, and insert it into the blockchain by spending the coins using the Link protocol. I’ve not yet calculated what the cost of a message will be, but I do estimate you can do several hundred thousand torrents for 1 FTC.
[/quote]So are we just storing torrent files in the blockchain? Torrents are notoriously resistant to adaptation. There is no torrent modify protocol or dynamic torrent. For example, there is no torrent proxy which can turn a folder being shared into a torrent file in real time.
I’ll work on FLUX and see how it fits here.
-
[quote name=“zerodrama” post=“41185” timestamp=“1386429718”]
[quote author=Kevlar link=topic=5558.msg41054#msg41054 date=1386410677]
[quote author=Vidicus link=topic=5558.msg41021#msg41021 date=1386407620]
I would match someone up to 100 FTC for this project and some of my personal time. I have some computer programming experience (pascal :o , Visual Basic, and Python) but am definitely rusty. What would the python have to do exactly?
EDIT: Sent 5.666 FTC to your sig address Kevlar, hopefully that gets a few more lines of code into this project!
[/quote]I’m already hard at work on it. Your donations are GREATLY appreaciated. :)
The python would have to read the pirate bay database, and insert it into the blockchain by spending the coins using the Link protocol. I’ve not yet calculated what the cost of a message will be, but I do estimate you can do several hundred thousand torrents for 1 FTC.
[/quote]So are we just storing torrent files in the blockchain? Torrents are notoriously resistant to adaptation. There is no torrent modify protocol or dynamic torrent. For example, there is no torrent proxy which can turn a folder being shared into a torrent file in real time.
I’ll work on FLUX and see how it fits here.
[/quote]That’s one possibility and you can do that with the Link protocol but I’m working on something a little more efficient in terms of storage. Why store the whole torrent when you can store a tiny fraction and accomplish the same thing?
As for modifying torrents, torrents are like URLs: you don’t modify one, you use a new one. That’s an important feature, because it means the same data will always be downloaded for a given torrent. Want different data? You wanted a different torrent.
Making them a directory at a time is trivial.
-
Is there any way the concept of holding the magnet files can encourage or push people to use i2p for further security?
I’m just envisioning people running the Link through Tor and going “Huh! It’s good enough for that. Let me go ahead run my torrent client through it.” and having a small nightmare about it.
-
[quote name=“Kevlar” post=“41250” timestamp=“1386439601”]
That’s one possibility and you can do that with the Link protocol but I’m working on something a little more efficient in terms of storage. Why store the whole torrent when you can store a tiny fraction and accomplish the same thing?
[/quote]See that’s what I was hoping for. Because many torrents are actually collections of files. This makes more sense. I still think link could be more dynamic. Like forking a linkchain for different remixes. Or updates to documents in progress.
[quote]
As for modifying torrents, torrents are like URLs: you don’t modify one, you use a new one. That’s an important feature, because it means the same data will always be downloaded for a given torrent. Want different data? You wanted a different torrent.
[/quote]Except we also have urls which are actually queries formatted as directories for search engines to crawl, which would mean blockchain search engines.
[quote]
Making them a directory at a time is trivial.
[/quote]I really should just build this feature and stop whining about it.
-
[quote name=“HopeStillFlies” post=“41254” timestamp=“1386440056”]
Is there any way the concept of holding the magnet files can encourage or push people to use i2p for further security?I’m just envisioning people running the Link through Tor and going “Huh! It’s good enough for that. Let me go ahead run my torrent client through it.” and having a small nightmare about it.
[/quote]Well they can run the blockchain download through the Tor network. That’s easy to do. I personally run my uTorrent over Tor and while I won’t max my bandwidth that way, it works well enough.
-
Ok, I’ve made a few changes to the initial pass. Specifically I’ve changed it from a fixed-length block-based encoding scheme to a variable-length stream-based encoding scheme based on op-codes to maximize compression and minimize wasted space in the blockchain.
Now is the time to get your op-codes submitted for inclusion. The list is far from complete, but please read the description of the ones suggested so far and if you have any ideas for other ones please let me know!
-
Is there anything someone who doesn’t have cryptos to donate, and doesn’t have the underlying computer skills, can do to help? It think this idea is brilliant.
-
[quote name=“Kevlar” post=“41313” timestamp=“1386448154”]
Ok, I’ve made a few changes to the initial pass. Specifically I’ve changed it from a fixed-length block-based encoding scheme to a variable-length stream-based encoding scheme based on op-codes to maximize compression and minimize wasted space in the blockchain.Now is the time to get your op-codes submitted for inclusion. The list is far from complete, but please read the description of the ones suggested so far and if you have any ideas for other ones please let me know!
[url=https://github.com/TSavo/Link/blob/master/README.md]https://github.com/TSavo/Link/blob/master/README.md[/url]
[/quote]I would do this for opcodes:
1st byte = module slot
2nd byte = module functionSo what you have defined so far is module 0 - the core link module. I would move 10, 11, FE, FF to 0A, 0B, 0D, 0E.
Module function 0 for all modules is the module key. Module function F for all modules is the extension range. Function 0 would identify the module we expect to be installed there. Function F would allow us to extend a module over several slots.The module key could be a digest of module name, version, VFW Post, geocaching GPS info, and other identifiers - they wouldn’t need to be listed, just the digest would need to match. The extension range would tell us the beginning and ending module slot which are allocated.
So core module/module key = Opcode 00. The extension key just points to module slot 0 as beginning and end. Say module 1 is a track list and module 2 is a list of ratings. Then track list/module key = Opcode 10 and ratings list/module key = Opcode 20. Both would also have extension range with beginning and end 1,1 and 2,2.
This means 14 functions per module slot. It allows for extreme extending of any concepts we want to add without getting lost. Core module in slot 0 is reserved for basic link protocol. I would suggest reserving slot F as well. The rest can be used by extenders. That leaves 14 slots of 14 functions each, for whatever we want to do. There is some duplication of info, but that may be the best way to do it.
If this isn’t too weird, I’ll set up FLUX (Flux Link Unit eXchange) functions within this same scheme.
One thing I would add (Peter Falk voice): some of these functions may require read write capability, but that’s for another time.
One favor Kevlar, I want to be able to read these streams from nodejs just for correctness in encoding. I might need to check with you on that.
-
[quote name=“zerodrama” post=“41331” timestamp=“1386453754”]
I would do this for opcodes:
1st byte = module slot
2nd byte = module function
[/quote]Sounds good. I’ll start organizing them next time I’m in the doc.
[quote]
One thing I would add (Peter Falk voice): some of these functions may require read write capability, but that’s for another time.
[/quote]Yeah, I’ve thought about a version scheme. It wouldn’t be hard. You could just say that this transaction succeeds this prior transaction, and sign the whole thing for security so only intended authors could version.
[quote]
One favor Kevlar, I want to be able to read these streams from nodejs just for correctness in encoding. I might need to check with you on that.
[/quote]Yep, I’m probably going to be doing the same in my initial pass. I’ve not decided on a language just yet. I’ll be sure to post anything nodish. Sails.js is lookin’ pretty rockin.
-
[quote name=“Kevlar” post=“41346” timestamp=“1386455714”]
[quote]
One thing I would add (Peter Falk voice): some of these functions may require read write capability, but that’s for another time.
[/quote]Yeah, I’ve thought about a version scheme. It wouldn’t be hard. You could just say that this transaction succeeds this prior transaction, and sign the whole thing for security so only intended authors could version.
[/quote]I’m gonna experiment with soft-reorgs here. If it works we can expand it to end 51% attacks. This would allow a safe environment for large and small currencies unencumbered by the philosophies of big players or unreasonable devs (ONE TROO COIN).
[quote]
[quote]
One favor Kevlar, I want to be able to read these streams from nodejs just for correctness in encoding. I might need to check with you on that.
[/quote]Yep, I’m probably going to be doing the same in my initial pass. I’ve not decided on a language just yet. I’ll be sure to post anything nodish. Sails.js is lookin’ pretty rockin.
[/quote]I normally hate frameworks (I want to know everything lol), but that looks very nice.
-
[quote name=“zerodrama” post=“41368” timestamp=“1386460296”]
I’m gonna experiment with soft-reorgs here. If it works we can expand it to end 51% attacks. This would allow a safe environment for large and small currencies unencumbered by the philosophies of big players or unreasonable devs (ONE TROO COIN).
[/quote]So, this could mean an end to ACP?
-
[quote name=“Tuck Fheman” post=“41383” timestamp=“1386463986”]
[quote author=zerodrama link=topic=5558.msg41368#msg41368 date=1386460296]I’m gonna experiment with soft-reorgs here. If it works we can expand it to end 51% attacks. This would allow a safe environment for large and small currencies unencumbered by the philosophies of big players or unreasonable devs (ONE TROO COIN).
[/quote]So, this could mean an end to ACP?
[/quote]The danger from 51% comes from two places:
1. 2 options - accepted or rejected - this gives us the > 50% value. FIX: create partial transactions, leading to more options, which will then require n-1/n of the network to override given n options.
2. no barrier once you cross the threshold - this means reorgs are free. FIX: make reorgs cost processing power#1 is a logical puzzle and will take some sideways thinking
#2 can be implemented as a feature and generalizedACP is a crutch. It will be removed eventually.
There are a couple more subtle reasons. But we’ll leave those for later.
