|
|
| (638 intermediate revisions by more than 100 users not shown) |
| Line 1: |
Line 1: |
| {{lowercase}}
| | This is a preview for the new '''MathML rendering mode''' (with SVG fallback), which is availble in production for registered users. |
| {{Infobox software
| |
| | name = rsync
| |
| | logo = [[File:Newrsynclogo.png|160px]]
| |
| | screenshot = <!-- [[File: ]] -->
| |
| | caption = rsync logo
| |
| | collapsible =
| |
| | author = [[Andrew Tridgell]], [[Paul Mackerras]]
| |
| | developer = [[Wayne Davison]]
| |
| | released = {{Start date|1996|06|19}}<ref name=rsyncrel>{{cite newsgroup | newsgroup = comp.os.linux.announce | date = 19 June 1996 | first = Andrew | last = Tridgell | url = http://groups.google.com/group/comp.os.linux.announce/msg/3bb93f6484065f20 | title = First release of rsync - rcp replacement | id = <cola-liw-835153950-21793-0@liw.clinet.fi>#1/1 | accessdate = 2007-07-19 }}</ref>
| |
| | discontinued =
| |
| | latest preview version =
| |
| | latest preview date = <!-- {{start date and age|YYYY|MM|DD}} -->
| |
| | frequently updated = yes
| |
| | programming language = [[C (programming language)|C]]
| |
| | operating system =
| |
| | platform = [[Unix-like]], [[Windows]]
| |
| | size =
| |
| | language =
| |
| | status = active
| |
| | genre = [[Data transfer]], [[Differential backup]]
| |
| | license = [[GNU General Public License#Version 3|GNU GPLv3]]
| |
| | website = {{URL|http://rsync.samba.org}}
| |
| }}
| |
| '''rsync''' is a [[software application]] and [[network protocol]] for [[Unix-like]] systems with ports to [[Windows]] that synchronizes [[computer file|file]]s and [[directory (file systems)|directories]] from one location to another while minimizing [[data]] transfer by using [[delta encoding]] when appropriate. Quoting the official [http://rsync.samba.org website]: "rsync is a file transfer program for Unix systems. rsync uses the 'rsync algorithm' which provides a very fast method for bringing remote files into sync."<ref>[http://rsync.samba.org/features.html rsync features], Retrieved 29 Jul. 2012.</ref> An important feature of rsync not found in most similar programs/protocols is that the [[mirror (computing)|mirror]]ing takes place with only one transmission in each direction. rsync can copy or display directory contents and copy files, optionally using [[data compression|compression]] and [[recursion]]. | |
|
| |
|
| In [[Daemon (computer software)|daemon]] mode, rsync listens on the default [[Transmission Control Protocol|TCP]] [[TCP and UDP port|port]] of 873, serving files in the native rsync protocol or via a remote [[shell (computing)|shell]] such as [[Remote Shell|RSH]] or [[Secure Shell|SSH]]. In the latter case, the rsync client executable must be installed on the remote machine as well as on the local machine.
| | If you would like use the '''MathML''' rendering mode, you need a wikipedia user account that can be registered here [[https://en.wikipedia.org/wiki/Special:UserLogin/signup]] |
| | * Only registered users will be able to execute this rendering mode. |
| | * Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere. |
|
| |
|
| Released under the [[GNU General Public License#Version 3|GNU General Public License version 3]], rsync is [[free software]]. It is widely used.<ref>[http://books.google.com/books?id=Duz1wQEBkb8C&pg=PA280&dq=rsync+widely+used&cd=1#v=onepage&q=rsync%20widely%20used&f=false Lossless compression handbook]</ref><ref>[http://books.google.com/books?id=s8oZF1_ZJTQC&pg=PA316&dq=rsync+widely+used&cd=6#v=onepage&q=rsync%20widely%20used&f=false Web content caching and distribution: proceedings of the 8th International Workshop]</ref><ref>[http://hssl.cs.jhu.edu/ip-rsync/ip-rsync.pdf In-Place Rsync: File Synchronization for Mobile and Wireless Devices], David Rasch and Randal Burns, Department of Computer Science ,Johns Hopkins University</ref><ref>{{cite journal | id = {{citeseerx|10.1.1.95.5042}} | title = Towards an Efficient, Scalable Replication Mechanism for the I2-DSI Project | first1 = Bert J. | last1 = Dempsey | first2 = Debra | last2 = Weiss | date = April 30, 1999 | journal = Technical Report TR-1999-01 }}</ref>
| | Registered users will be able to choose between the following three rendering modes: |
|
| |
|
| == History ==
| | '''MathML''' |
| [[Andrew Tridgell]] and [[Paul Mackerras]] wrote the original rsync. Tridgell discusses the design, implementation and performance of rsync in chapters 3 through 5 of his [[Australian National University]] [[Doctor of Philosophy|Ph.D.]] thesis.<ref>Andrew Tridgell: [http://samba.org/~tridge/phd_thesis.pdf Efficient Algorithms for Sorting and Synchronization], February 1999. Retrieved 29 Sept. 2009.</ref>
| | :<math forcemathmode="mathml">E=mc^2</math> |
|
| |
|
| rsync was first announced on 19 June 1996.<ref name=rsyncrel /> Rsync 3.0 was released on 1 March 2008.<ref>{{cite mailing list | url = http://lists.samba.org/archive/rsync-announce/2008/000057.html | date = 1 March 2008 | first = Wayne | last = Davison | title = Rsync 3.0.0 released | mailinglist = rsync-announce }}</ref>
| | <!--'''PNG''' (currently default in production) |
| | :<math forcemathmode="png">E=mc^2</math> |
|
| |
|
| == Uses ==
| | '''source''' |
| '''rsync''' was originally written as a replacement for [[rcp (Unix)|'''rcp''']] and [[Secure copy|'''scp''']]. As such, it has a similar syntax to its parent programs.<ref>See the [http://rsync.samba.org/ftp/rsync/README README file]</ref> Like its predecessors, it still requires a source and a destination to be specified, either of which may be remote. Because of the flexibility, speed and scriptability of rsync, it has become a standard Linux utility and is included in all popular Linux distributions. As a result, rsync has been ported to Windows (via commercial [[Cygwin]], freeware [[Grsync]] or [[Windows Services for UNIX|SFU]]<ref>http://www.suacommunity.com/tool_warehouse.aspx</ref>) and Mac OS. | | :<math forcemathmode="source">E=mc^2</math> --> |
|
| |
|
| Possible uses:
| | <span style="color: red">Follow this [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering link] to change your Math rendering settings.</span> You can also add a [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering-skin Custom CSS] to force the MathML/SVG rendering or select different font families. See [https://www.mediawiki.org/wiki/Extension:Math#CSS_for_the_MathML_with_SVG_fallback_mode these examples]. |
| <source lang="bash"> | |
| rsync [OPTION] … SRC [SRC] … [USER@]HOST:DEST
| |
| rsync [OPTION] … [USER@]HOST:SRC [DEST]
| |
| </source>
| |
|
| |
|
| One of the earliest applications of rsync was to implement mirroring or backup for multiple Unix clients to a central Unix server using rsync/ssh and standard Unix accounts.
| | ==Demos== |
|
| |
|
| With a scheduling utility such as [[cron]], one can schedule automated encrypted rsync-based mirroring between multiple hosts and a central server.
| | Here are some [https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Frederic.wang demos]: |
|
| |
|
| == Examples ==
| |
|
| |
|
| A command line to mirror [[FreeBSD]] might look like:
| | * accessibility: |
| | ** Safari + VoiceOver: [https://commons.wikimedia.org/wiki/File:VoiceOver-Mac-Safari.ogv video only], [[File:Voiceover-mathml-example-1.wav|thumb|Voiceover-mathml-example-1]], [[File:Voiceover-mathml-example-2.wav|thumb|Voiceover-mathml-example-2]], [[File:Voiceover-mathml-example-3.wav|thumb|Voiceover-mathml-example-3]], [[File:Voiceover-mathml-example-4.wav|thumb|Voiceover-mathml-example-4]], [[File:Voiceover-mathml-example-5.wav|thumb|Voiceover-mathml-example-5]], [[File:Voiceover-mathml-example-6.wav|thumb|Voiceover-mathml-example-6]], [[File:Voiceover-mathml-example-7.wav|thumb|Voiceover-mathml-example-7]] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Audio-Windows7-InternetExplorer.ogg Internet Explorer + MathPlayer (audio)] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-SynchronizedHighlighting-WIndows7-InternetExplorer.png Internet Explorer + MathPlayer (synchronized highlighting)] |
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Braille-Windows7-InternetExplorer.png Internet Explorer + MathPlayer (braille)] |
| | ** NVDA+MathPlayer: [[File:Nvda-mathml-example-1.wav|thumb|Nvda-mathml-example-1]], [[File:Nvda-mathml-example-2.wav|thumb|Nvda-mathml-example-2]], [[File:Nvda-mathml-example-3.wav|thumb|Nvda-mathml-example-3]], [[File:Nvda-mathml-example-4.wav|thumb|Nvda-mathml-example-4]], [[File:Nvda-mathml-example-5.wav|thumb|Nvda-mathml-example-5]], [[File:Nvda-mathml-example-6.wav|thumb|Nvda-mathml-example-6]], [[File:Nvda-mathml-example-7.wav|thumb|Nvda-mathml-example-7]]. |
| | ** Orca: There is ongoing work, but no support at all at the moment [[File:Orca-mathml-example-1.wav|thumb|Orca-mathml-example-1]], [[File:Orca-mathml-example-2.wav|thumb|Orca-mathml-example-2]], [[File:Orca-mathml-example-3.wav|thumb|Orca-mathml-example-3]], [[File:Orca-mathml-example-4.wav|thumb|Orca-mathml-example-4]], [[File:Orca-mathml-example-5.wav|thumb|Orca-mathml-example-5]], [[File:Orca-mathml-example-6.wav|thumb|Orca-mathml-example-6]], [[File:Orca-mathml-example-7.wav|thumb|Orca-mathml-example-7]]. |
| | ** From our testing, ChromeVox and JAWS are not able to read the formulas generated by the MathML mode. |
|
| |
|
| % rsync -avz --delete ftp4.de.FreeBSD.org::FreeBSD/ /pub/FreeBSD/<ref>[http://www.freebsd.org/doc/en/articles/hubs/mirror-howto.html How to Mirror FreeBSD (With rsync)]</ref>
| | ==Test pages == |
|
| |
|
| The [[Apache HTTP Server]] supports only rsync for updating mirrors.
| | To test the '''MathML''', '''PNG''', and '''source''' rendering modes, please go to one of the following test pages: |
| | *[[Displaystyle]] |
| | *[[MathAxisAlignment]] |
| | *[[Styling]] |
| | *[[Linebreaking]] |
| | *[[Unique Ids]] |
| | *[[Help:Formula]] |
|
| |
|
| rsync -avz --delete --safe-links rsync.apache.org::apache-dist /path/to/mirror<ref>[http://www.apache.org/info/how-to-mirror.html How to become a mirror for the Apache Software Foundation]</ref>
| | *[[Inputtypes|Inputtypes (private Wikis only)]] |
| | | *[[Url2Image|Url2Image (private Wikis only)]] |
| The preferred (and simplest) way to mirror the [[PuTTY]] website to the current directory is to use rsync.
| | ==Bug reporting== |
| | | If you find any bugs, please report them at [https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=Math&version=master&short_desc=Math-preview%20rendering%20problem Bugzilla], or write an email to math_bugs (at) ckurs (dot) de . |
| rsync -auH rsync://rsync.chiark.greenend.org.uk/ftp/users/sgtatham/putty-website-mirror/ .<ref>[http://www.chiark.greenend.org.uk/~sgtatham/putty/mirrors.html#guidelines PuTTY Web Site Mirrors: Mirroring guidelines]</ref>
| |
| | |
| A way to mimic the capabilities of [[Time Machine (Mac OS)]].
| |
| | |
| #date=`date "+%Y-%m-%dT%H:%M:%S"`
| |
| date=`date "+%FT%T"`
| |
| rsync -aP --link-dest=$HOME/Backups/current /path/to/important_files $HOME/Backups/back-$date
| |
| rm -f $HOME/Backups/current
| |
| ln -s $HOME/Backups/back-$date $HOME/Backups/current<ref>[http://blog.interlinked.org/tutorials/rsync_time_machine.html Rsync setup to run like Time Machine]</ref>
| |
| | |
| == Algorithm ==
| |
| The rsync utility uses an [[algorithm]] invented by the Australian computer programmer [[Andrew Tridgell]] for efficiently transmitting a structure (such as a file) across a communications link when the receiving computer already has a similar, but not identical, version of the same structure.
| |
| | |
| The recipient splits its copy of the file into fixed-size non-overlapping chunks and computes two [[checksum]]s for each chunk: the [[MD5]] [[hash function|hash]], and a weaker '[[Rolling hash|rolling checksum]]'. (Prior to version 30 of the protocol, released with rsync version 3.0.0, it used [[MD4]] hashes rather than MD5.<ref>[http://rsync.samba.org/ftp/rsync/src/rsync-3.0.0-NEWS NEWS for rsync 3.0.0] (1 Mar 2008)</ref>) It sends these checksums to the sender.
| |
| | |
| The sender computes the rolling checksum for every chunk of size <math>S</math> in its own version of the file, even overlapping chunks. This can be calculated efficiently because of a special property of the rolling checksum: if the rolling checksum of [[byte]]s <math>n</math> through <math>n+S-1</math> is <math>R</math>, the rolling checksum of bytes <math>n+1</math> through <math>n+S</math> can be computed from <math>R</math>, byte <math>n</math>, and byte <math>n+S</math> without having to examine the intervening bytes. Thus, if one had already calculated the rolling checksum of bytes 1–25, one could calculate the rolling checksum of bytes 2–26 solely from the previous checksum (R), byte 1 (n), and byte 26 (n+S).
| |
| | |
| The [[rolling hash|rolling checksum]] used in rsync is based on Mark Adler's [[adler-32]] checksum, which is used in [[zlib]], and is itself based on [[Fletcher's checksum]].
| |
| | |
| The sender then compares its rolling checksums with the set sent by the recipient to determine if any matches exist. If they do, it verifies the match by computing the hash for the matching [[Block (programming)|block]] and by comparing it with the hash for that block sent by the recipient.
| |
| | |
| The sender then sends the recipient those parts of its file that did not match the recipient's blocks, along with information on where to merge these blocks into the recipient's version. This makes the copies identical. However, there is a small probability that differences between chunks in the sender and recipient are not detected, and thus remains uncorrected. This requires a simultaneous hash collision in MD5 and the rolling checksum. It is possible to generate MD5 collisions, and the rolling checksum is not cryptographically strong, but the chance for this to occur by accident is nevertheless extremely remote. With 128 bits from MD5 plus 32 bits from the rolling checksum, and assuming maximum [[entropy (information theory)|entropy]] in these bits, the probability of a hash collision with this combined checksum is 2<sup>−(128+32)</sup> = 2<sup>−160</sup>. The actual probability is a few times higher, since good checksums approach maximum output entropy but very rarely achieve it.
| |
| | |
| If the sender's and recipient's versions of the file have many sections in common, the utility needs to transfer relatively little data to synchronize the files. Note that if usual [[data compression]] algorithms are used, files that are similar when uncompressed may be very different when compressed, and thus the entire file will need to be transferred – local changes in uncompressed files yield global changes in compressed files. This is particularly an issue with mirroring of [[archive file]]s, such as [[disk image]]s and compressed [[tar (file format)|tarballs]], where often individual files change. Some compression programs, such as [[gzip]] and [[bzip2]], provide a special "rsyncable" mode which allows these files to be efficiently rsynced, by ensuring that local changes in the uncompressed file yields only local changes in the compressed file.
| |
| | |
| While the rsync algorithm forms the heart of the rsync application that essentially optimizes transfers between two computers over TCP/IP, the rsync application supports other key features that aid significantly in data transfers or backup. They include compression and decompression of data block by block using [[zlib]] at sending and receiving ends, and support for protocols such as [[Secure Shell|ssh]] that enables encrypted transmission of compressed and efficient differential data using rsync algorithm. Instead of ssh, [[stunnel]] can also be used to create an encrypted tunnel to secure the data transmitted.
| |
| | |
| Finally, rsync is capable of limiting the bandwidth consumed during a transfer, a useful feature that few other standard file transfer programs offer.
| |
| | |
| == Variations == | |
| A utility called '''{{visible anchor|rdiff}}''' uses the rsync algorithm to generate [[Delta encoding|delta file]]s with the difference from file A to file B (like the utility [[diff]], but in a different delta format). The delta file can then be applied to file A, turning it into file B (similar to the [[patch (Unix)|patch]] utility).
| |
| | |
| Unlike diff, the process of creating a delta file has two steps: first a signature file is created from file A, and then this (relatively small) signature and file B are used to create the delta file. Also unlike diff, rdiff works well with [[binary file]]s.
| |
| | |
| Using rdiff, a utility called ''rdiff-backup'' has been created, capable of maintaining a [[backup]] mirror of a file or directory either locally or remotely over the network, on another server. rdiff-backup stores incremental rdiff deltas with the backup, with which it is possible to recreate any backup point.<ref>{{official website
| |
| | 2 = rdiff-backup
| |
| | 1 = www.nongnu.org/rdiff-backup/
| |
| }}</ref>
| |
| | |
| [[Duplicity (software)|Duplicity]] is a variation on rdiff-backup that allows for backups without cooperation from the storage server, as with simple storage services like [[Amazon S3]]. It works by generating the hashes for each block in advance, encrypting them, and storing them on the server, then retrieving them when doing an incremental backup. The rest of the data is also stored encrypted for security purposes.
| |
| | |
| [[rsyncrypto]] is a utility to encrypt files in an rsync-friendly fashion. The rsyncrypto algorithm ensures that two almost identical files, when encrypted with rsyncrypto and the same key, will produce almost identical encrypted files. This allows for the low-overhead data transfer achieved by rsync while providing encryption for secure transfer and storage of sensitive data in a remote location.<ref>{{official website
| |
| | 2 = rsyncrypto
| |
| | 1 = rsyncrypto.lingnu.com/
| |
| }}</ref>
| |
| | |
| An alternative to manually scripting rsync is the Free Software (FLOSS) GUI program [[BackupPC]], which performs automatic scheduled backups to rsync servers.
| |
| | |
| As of Mac OS X 10.5 and later, there is a special -E or --extended-attributes switch which allows retaining much of the [[Hierarchical File System|HFS]] file metadata when syncing between two machines supporting this feature. This is achieved by transmitting the proprietary [[Resource Fork]] along with the Data Fork.<ref>http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/rsync.1.html</ref>
| |
| | |
| == Practical applications == | |
| '''rsync''' can be used as a method to intelligently copy or back files up from one location to another. For example, within a music library, all music files may be located within an artist folder, with an album subdirectory. If you have an external hard drive that serves as a backup of your music then manually updating the music, even when copies merge directories, can be an error prone and time consuming task best left to automation. However, rsync can be used to scan all of the files in your music library, as well as the subdirectories, and to add only the ones that are not present on the external hard drive.
| |
| | |
| == Solutions using Rsync ==
| |
| | |
| {|class="wikitable sortable" border="1"
| |
| |-
| |
| ! Name !! Linux || Mac OS || Windows || Comments
| |
| |-
| |
| | [[BackupAssist]] || {{No}}|| {{No}}|| {{Yes}} || Direct mirror or with history, VSS
| |
| |-
| |
| | [http://backintime.le-web.org/ Back In Time] || {{Yes}}||{{No}}||{{No}} ||
| |
| |-
| |
| | [[Cwrsync]] || {{No}}|| {{No}}|| {{Yes}} || Rsync for Windows (based on [[Cygwin]])
| |
| |-
| |
| | [[CarbonCopyCloner]] || {{No}}|| {{Yes}}|| {{No}} || Local whole disk backup
| |
| |-
| |
| | [[DSynchronize]] || {{No}}|| {{No}}|| {{Yes}} ||
| |
| |-
| |
| | [[LuckyBackup]] || {{Yes}}|| {{Yes}}|| {{Yes}} ||
| |
| |-
| |
| | gadmin-rsync || {{Yes}}||{{No}} || {{No}}|| Part of [[Gadmintools]]
| |
| |-
| |
| | [[Grsync]] || {{Yes}}|| {{Yes}}|| {{Yes}}||
| |
| |-
| |
| | [[QtdSync]] || {{Yes}}|| {{No}}|| {{Yes}}||
| |
| |-
| |
| | [[PureSync]] || {{No}}|| {{No}}|| {{Yes}}||
| |
| |-
| |
| | [[DeltaCopy]] || {{No}}|| {{No}}|| {{Yes}}|| Open Source, Free - [http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp WebSite] - [http://www.aboutmyx.com/files/DeltaCopy.zip Download]
| |
| |-
| |
| | [[Yintersync]] || {{No}}|| {{No}}|| {{Yes}}|| VSS Shadow Copies, Email Reports, Scheduler, NTFS Permissions, Centralised.
| |
| |-
| |
| | [[Syncrify]] || {{Yes}}|| {{No}}|| {{Yes}}||
| |
| |-
| |
| | [[Backuplist+]] || {{No}}|| {{Yes}}|| {{No}}||
| |
| |-
| |
| | [[RipCord Backup]] || {{No}}|| {{Yes}}|| {{No}}||
| |
| |-
| |
| | [[RsyncX]] || {{No}}|| {{Yes}}|| {{No}}||
| |
| |-
| |
| | [[arRsync]] || {{No}}|| {{Yes}}|| {{No}}||
| |
| |-
| |
| | [[Duplicati (software)|Duplicati]] || {{Yes}}|| {{Yes}}|| {{Yes}}|| VSS, LVM snapshots, Scheduler
| |
| |-
| |
| | [[FolderWatch]] || {{No}}|| {{Yes}}|| {{No}}|| Supports real-time and on-demand syncing
| |
| |-
| |
| | [[rdiff-backup]] || {{Yes}}|| {{Yes}}|| {{Yes}}|| supports history
| |
| |-
| |
| | [[Get Backup]] || {{No}}|| {{Yes}}|| {{No}}|| Partial sync, comparison between sync, scheduler, disk cloning
| |
| |-
| |
| | [[Unison (file synchronizer)|Unison]] || {{Yes}} || {{Yes}} || {{Yes}} || File synchronizer using Rsync algorithm
| |
| |-
| |
| |}
| |
| | |
| == See also ==
| |
| {{Portal|Free software}}
| |
| | |
| * [[Remote Differential Compression]]
| |
| | |
| ==References==
| |
| {{Reflist}}
| |
| | |
| ==External links==
| |
| * {{Official website|http://rsync.samba.org}}
| |
| * [http://rsync.samba.org/tech_report/ rsync algorithm]
| |
| * [http://rsync.samba.org/examples.html Official rsync examples]
| |
| | |
| [[Category:1996 software]]
| |
| [[Category:Data synchronization]]
| |
| [[Category:Free backup software]]
| |
| [[Category:Free network-related software]]
| |
| [[Category:Networking algorithms]]
| |
| [[Category:Network file transfer protocols]]
| |
| [[Category:Unix network-related software]]
| |
| [[Category:Free file transfer software]]
| |
| | |
| [[ca:Rsync]]
| |
| [[cs:Rsync]]
| |
| [[da:Rsync]]
| |
| [[de:Rsync]]
| |
| [[es:Rsync]]
| |
| [[fr:Rsync]]
| |
| [[it:Rsync]]
| |
| [[nl:Rsync]]
| |
| [[ja:Rsync]]
| |
| [[pl:Rsync]]
| |
| [[pt:Rsync]]
| |
| [[ru:Rsync]]
| |
| [[sv:Rsync]]
| |
| [[tr:Rsync]]
| |
| [[zh:Rsync]]
| |
