Original release 20160328

[fex.git] / doc / concept

diff --git a/doc/concept b/doc/concept
index ead53878c1dff9d91c1033cc703f8a790c21925e..fde990246e95755f6588e603401afad80be72658 100644 (file)
--- a/doc/concept
+++ b/doc/concept
@@ -17,7 +17,7 @@ Main features of F*EX
   * full-users can create subusers, who can send only to this full-user
   * full-users can create groups, an analogy to mailing lists, but for files
   * self registration possible for internal and external users
   * full-users can create subusers, who can send only to this full-user
   * full-users can create groups, an analogy to mailing lists, but for files
   * self registration possible for internal and external users

-  * maintenance-free: no admin interaction necessary 
+  * maintenance-free: no admin interaction necessary

   * sending to multiple recipients needs storage on the server only once
   * quotas for sending and receiving
   * F*EX is a HTTP web-service and needs no firewall-tunnels
   * sending to multiple recipients needs storage on the server only once
   * quotas for sending and receiving
   * F*EX is a HTTP web-service and needs no firewall-tunnels


@@ -40,7 +40,7 @@ Main features of F*EX
 
 The end user normally uses F*EX with his web browser and the URLs
 http://YOURFEXSERVER/fup (file upload) and http://YOURFEXSERVER/fop (file
 
 The end user normally uses F*EX with his web browser and the URLs
 http://YOURFEXSERVER/fup (file upload) and http://YOURFEXSERVER/fop (file

-download). 
+download).

 
 
 F*EX is not an anonymous service (exception: "public upload" and
 
 
 F*EX is not an anonymous service (exception: "public upload" and


@@ -52,7 +52,7 @@ program "fac" (F*EX admin control) or http://YOURFEXSERVER/fac
 Alternativly the users can register theirselves with http://YOURFEXSERVER/fur
 (F*EX user registration), if the admin allows them to do so. This is done by
 setting the variables @local_domains and @local_hosts in FEXHOME/lib/fex.ph
 Alternativly the users can register theirselves with http://YOURFEXSERVER/fur
 (F*EX user registration), if the admin allows them to do so. This is done by
 setting the variables @local_domains and @local_hosts in FEXHOME/lib/fex.ph

-Example: 
+Example:

 
        @local_hosts = qw(127.0.0.1 10.10.100.0-10.10.255.255);
        @local_domains = qw(rus.uni-stuttgart.de flupp.org);
 
        @local_hosts = qw(127.0.0.1 10.10.100.0-10.10.255.255);
        @local_domains = qw(rus.uni-stuttgart.de flupp.org);


@@ -75,12 +75,12 @@ the local recipient domains. For example:
 
 
 F*EX full users can create one time upload URLs with
 
 
 F*EX full users can create one time upload URLs with

-http://YOURFEXSERVER/fuc 
+http://YOURFEXSERVER/fuc

 
 With such a URL a foreign user can send this F*EX full user a single file.
 
 F*EX full users can theirselves register "subusers" with
 
 With such a URL a foreign user can send this F*EX full user a single file.
 
 F*EX full users can theirselves register "subusers" with

-http://YOURFEXSERVER/fuc 
+http://YOURFEXSERVER/fuc

 
 Subusers can only fex to their full-user, not to any others, and they
 cannot create other subusers.
 
 Subusers can only fex to their full-user, not to any others, and they
 cannot create other subusers.


@@ -95,7 +95,7 @@ he cannot change anymore his default settings (KEEP, AUTODELETE, etc)
 
 
 The F*EX user is the same as the "sender" in the fup CGI and the "from"
 
 
 The F*EX user is the same as the "sender" in the fup CGI and the "from"

-parameter in the F*EX URL. 
+parameter in the F*EX URL.

 
 The (confusing) naming scheme is historically based :-)
 
 
 The (confusing) naming scheme is historically based :-)
 


@@ -106,7 +106,7 @@ email.
 
 
 You do not need to build F*EX URLs manually, they are generated by the
 
 
 You do not need to build F*EX URLs manually, they are generated by the

-F*EX programs. 
+F*EX programs.

 
 
 A F*EX session is based on unique URL parameters or POST variables (FROM,
 
 
 A F*EX session is based on unique URL parameters or POST variables (FROM,


@@ -121,19 +121,18 @@ A SKEY is made of md5_hex("$mainuser:$subuser:$subuserid")
 A GKEY is made of md5_hex("$mainuser:$groupname:$groupuser:$groupuserid")
 
 Note: the AKEY, SKEY and GKEY always can be stolen by a network sniffer!
 A GKEY is made of md5_hex("$mainuser:$groupname:$groupuser:$groupuserid")
 
 Note: the AKEY, SKEY and GKEY always can be stolen by a network sniffer!

-If you need true security, then you have to use https (SSL) instead of
-http! 
+If you need true security, then you have to use https instead of http!

 
 After download the file will be deleted after a grace time of 1 minute.
 This grace time allows a recipient to get the file again if he had
 
 After download the file will be deleted after a grace time of 1 minute.
 This grace time allows a recipient to get the file again if he had

-problems in saving it. 
+problems in saving it.

 
 With the fexsend client the sender can change this behavior:
 
 option -D means "delay autodelete": do not delete the the file directly
 after download, but with the nightly fex_cleanup cronjob. More downloads
 are possible only from the same client (identified by cookie or ip
 
 With the fexsend client the sender can change this behavior:
 
 option -D means "delay autodelete": do not delete the the file directly
 after download, but with the nightly fex_cleanup cronjob. More downloads
 are possible only from the same client (identified by cookie or ip

-address). 
+address).

 
 option -K means "keep file": do not delete the file after download, but
 only after expiration date (normally 5 days). More downloads are possible
 
 option -K means "keep file": do not delete the file after download, but
 only after expiration date (normally 5 days). More downloads are possible


@@ -165,7 +164,7 @@ requires (HTTP) authorization. The credentials are the F*EX user email
 and auth-ID.
 See extra documentation SSL for using https.
 
 and auth-ID.
 See extra documentation SSL for using https.
 

-If you want to have encrypted emails, then you need a GPG public key for 
+If you want to have encrypted emails, then you need a GPG public key for

 the user fex. Create it with "gpg --gen-key" (use fex@YOURFEXSERVER as the
 user or the $bcc address from fex.ph if you have set it, see below). Next,
 the user has to upload his public key via webinterface "user config &
 the user fex. Create it with "gpg --gen-key" (use fex@YOURFEXSERVER as the
 user or the $bcc address from fex.ph if you have set it, see below). Next,
 the user has to upload his public key via webinterface "user config &


@@ -207,14 +206,14 @@ below!
 
 FEXHOME contains:
 
 
 FEXHOME contains:
 

-       spool/                          spool directory and user data 
+       spool/                          spool directory and user data

         htdocs/                                directory for generic download files
         bin/                           directory for programs
         cgi-bin/                       directory for CGI programs
        lib/                            directory for library and config files
         doc/                           additional documentation
         locale/                                alternative language installation
         htdocs/                                directory for generic download files
         bin/                           directory for programs
         cgi-bin/                       directory for CGI programs
        lib/                            directory for library and config files
         doc/                           additional documentation
         locale/                                alternative language installation

-       
+

 
 Files in spool:
 
 
 Files in spool:
 


@@ -241,12 +240,13 @@ Files in spool:
        $user/@KEEP                     keep default
        $user/@LOCALE                   locale default
        $user/@CAPTIVE                  user must not change his settings
        $user/@KEEP                     keep default
        $user/@LOCALE                   locale default
        $user/@CAPTIVE                  user must not change his settings

-       $user/@FEXYOURSELF              user can only fex to himself via 
+       $user/@FEXYOURSELF              user can only fex to himself via

                                         web interface
        $to/$from/$file/upload          file data in upload progress
        $to/$from/$file/filename        original file name
        $to/$from/$file/size            original file size
        $to/$from/$file/useragent       HTTP header User-Agent
                                         web interface
        $to/$from/$file/upload          file data in upload progress
        $to/$from/$file/filename        original file name
        $to/$from/$file/size            original file size
        $to/$from/$file/useragent       HTTP header User-Agent

+       $to/$from/$file/uurl            upload URL

        $to/$from/$file/data            file data after complete upload
        $to/$from/$file/keep            keep time (autoexpire) in days
        $to/$from/$file/autodelete      autodelete option: YES NO or DELAY
        $to/$from/$file/data            file data after complete upload
        $to/$from/$file/keep            keep time (autoexpire) in days
        $to/$from/$file/autodelete      autodelete option: YES NO or DELAY


@@ -294,7 +294,7 @@ This is done via UKEY, a unique upload identifier.
 The download key (DKEY) is a unique identifier for - guess what -
 downloading. It also prevents an attacker to get the file, because only
 the recipient knows the DKEY as part of the download URL from the
 The download key (DKEY) is a unique identifier for - guess what -
 downloading. It also prevents an attacker to get the file, because only
 the recipient knows the DKEY as part of the download URL from the

-notification email. 
+notification email.

 
 XKEY is an optional extra download key to have a short download URL in
 shape http://YOURFEXSERVER//XKEY
 
 XKEY is an optional extra download key to have a short download URL in
 shape http://YOURFEXSERVER//XKEY


@@ -302,7 +302,7 @@ The XKEY always starts with // and is provided as a COMMENT by the client.
 Example: fexsend schwuppdiwupp.jpg //
 The user has to realize that such URLs have very low security.
 
 Example: fexsend schwuppdiwupp.jpg //
 The user has to realize that such URLs have very low security.
 

-If you need to trace a F*EX request, then set 
+If you need to trace a F*EX request, then set

        $debug = 1;
 in fex.ph and look in ~/spool/.debug/ for the correspondening files.
 
        $debug = 1;
 in fex.ph and look in ~/spool/.debug/ for the correspondening files.
 


@@ -323,7 +323,7 @@ In this case "From: $admin" will be used in the notification emails.
 
 F*EX comes with its own web server: fexsrv
 Standard web servers like apache have been proven problematic, either in
 
 F*EX comes with its own web server: fexsrv
 Standard web servers like apache have been proven problematic, either in

-speed or because of a 2-GB-limit. 
+speed or because of a 2-GB-limit.

 
 It is not possible to use the F*EX CGIs with an alternative web server,
 because the F*EX CGIs need special fexsrv features.
 
 It is not possible to use the F*EX CGIs with an alternative web server,
 because the F*EX CGIs need special fexsrv features.


@@ -331,7 +331,7 @@ because the F*EX CGIs need special fexsrv features.
 xinetd starts fexsrv for every new connection, which then executes the
 CGIs fup (file upload), fop (file output), fuc (fex user control), foc
 (fex operation control), fac (fex admin control), fur (fex user
 xinetd starts fexsrv for every new connection, which then executes the
 CGIs fup (file upload), fop (file output), fuc (fex user control), foc
 (fex operation control), fac (fex admin control), fur (fex user

-registration) and sex (stream exchange). 
+registration) and sex (stream exchange).

 
 
 SEX has the opposite authorization model of FEX: The sender does not
 
 
 SEX has the opposite authorization model of FEX: The sender does not


@@ -342,7 +342,7 @@ otherwise. The client programs for sexing are sexsend and sexget. You can
 call them also fuckyou and fuckme :-)
 
 
 call them also fuckyou and fuckme :-)
 
 

-With the F*EX client fexsend you can have a streaming file transfer with 
+With the F*EX client fexsend you can have a streaming file transfer with

 spooling: on client side there is no temporary buffer file (archive), but
 the data is sent directly to the F*EX server. This saves time and disk
 space on the client.
 spooling: on client side there is no temporary buffer file (archive), but
 the data is sent directly to the F*EX server. This saves time and disk
 space on the client.


@@ -383,14 +383,16 @@ directory. You can name single IPs, also as IP ranges (example:
 129.69.0.0-129.69.255.255)
 
 dop is not a regular CGI program (like fup or fop), but a sub-program of
 129.69.0.0-129.69.255.255)
 
 dop is not a regular CGI program (like fup or fop), but a sub-program of

-fexsrv. 
+fexsrv.

 
 *.html files may contain $VARIABLES$ which will be substituted with the
 value of the corresponding environment variable. See example
 $SERVER_ADMIN$ in FEXHOME/htdocs/index.html
 
 *.html files may contain <<perl-code>> (even multiline) which will be
 
 *.html files may contain $VARIABLES$ which will be substituted with the
 value of the corresponding environment variable. See example
 $SERVER_ADMIN$ in FEXHOME/htdocs/index.html
 
 *.html files may contain <<perl-code>> (even multiline) which will be

-evaluated. See example FEXHOME/htdocs/dynamic.html
+evaluated and its output will be placed in. Same goes for <<<perl-code>>>
+but without output catching.
+See example FEXHOME/htdocs/dynamic.html

 This perl-code must not contain '>>' strings itself!
 Pay attention: do not place security relevant data inside << >> because it
 will be delivered to the client if the URL ends with '!'! See example:
 This perl-code must not contain '>>' strings itself!
 Pay attention: do not place security relevant data inside << >> because it
 will be delivered to the client if the URL ends with '!'! See example:


@@ -462,7 +464,7 @@ Or to delete inactive users you can put in fex.ph:
 $account_expire = "365:delete";
 
 This deletes user accounts automatically which have been inactive for 365
 $account_expire = "365:delete";
 
 This deletes user accounts automatically which have been inactive for 365

-days. 
+days.

 
 Of course you can use any number of days.
 
 
 Of course you can use any number of days.
 


@@ -489,7 +491,7 @@ because most proxies do not support persistant tcp sessions.
 See fex-client_2.pdf for the F*EX protocol specification.
 
 To understand and trace the F*EX protocol you can use fexsend with the -v
 See fex-client_2.pdf for the F*EX protocol specification.
 
 To understand and trace the F*EX protocol you can use fexsend with the -v

-option. 
+option.

 
 Example (--> means send to server, <-- means receive from server):
 
 
 Example (--> means send to server, <-- means receive from server):
 


@@ -504,47 +506,47 @@ TCPCONNECT to fex.rus.uni-stuttgart.de
 <-- X-SID: 8p2Y2qa2
 <-- X-Timeout: 30
 <-- Content-Length: 0
 <-- X-SID: 8p2Y2qa2
 <-- X-Timeout: 30
 <-- Content-Length: 0

-<-- 
+<--

 --> HEAD /fop/framstag@flupp.org/framstag@fex.rus.uni-stuttgart.de/X.png??ID=MD5H:226e896d0adab86892957aa8158b37ba HTTP/1.1
 -->
 <-- HTTP/1.1 200 OK
 <-- Content-Length: 0
 <-- X-Size: 0
 <-- X-Features: SID,CHECKRECIPIENT,GROUPS,QUOTA,FILEID,MULTIPOST,XKEY
 --> HEAD /fop/framstag@flupp.org/framstag@fex.rus.uni-stuttgart.de/X.png??ID=MD5H:226e896d0adab86892957aa8158b37ba HTTP/1.1
 -->
 <-- HTTP/1.1 200 OK
 <-- Content-Length: 0
 <-- X-Size: 0
 <-- X-Features: SID,CHECKRECIPIENT,GROUPS,QUOTA,FILEID,MULTIPOST,XKEY

-<-- 
+<--

 --> POST /fup HTTP/1.1
 --> Host: fex.rus.uni-stuttgart.de
 --> User-Agent: fexsend (Ubuntu 8.04.4 LTS)
 --> Content-Length: 149935162
 --> Content-Type: multipart/form-data; boundary=JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> POST /fup HTTP/1.1
 --> Host: fex.rus.uni-stuttgart.de
 --> User-Agent: fexsend (Ubuntu 8.04.4 LTS)
 --> Content-Length: 149935162
 --> Content-Type: multipart/form-data; boundary=JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs

---> 
+-->

 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="FROM"
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="FROM"

---> 
+-->

 --> framstag@fex.rus.uni-stuttgart.de
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="TO"
 --> framstag@fex.rus.uni-stuttgart.de
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="TO"

---> 
+-->

 --> framstag@flupp.org
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="ID"
 --> framstag@flupp.org
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="ID"

---> 
+-->

 --> MD5H:226e896d0adab86892957aa8158b37ba
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="FILESIZE"
 --> MD5H:226e896d0adab86892957aa8158b37ba
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="FILESIZE"

---> 
+-->

 --> 149934400
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="FILE"; filename="X.png"
 --> Content-Type: application/octet-stream
 --> Content-Length: 149934400
 --> X-File-ID: 1283077463
 --> 149934400
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
 --> Content-Disposition: form-data; name="FILE"; filename="X.png"
 --> Content-Type: application/octet-stream
 --> Content-Length: 149934400
 --> X-File-ID: 1283077463

---> 
+-->

 --> (file content)
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs--
 <-- Location: http://fex.rus.uni-stuttgart.de/fop/CoVhikzk/X.png
 <-- X-Recipient: framstag@flupp.org (autodelete=YES,keep=5)
 --> (file content)
 --> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs--
 <-- Location: http://fex.rus.uni-stuttgart.de/fop/CoVhikzk/X.png
 <-- X-Recipient: framstag@flupp.org (autodelete=YES,keep=5)

-<-- 
+<--

 
 Comment on the HEAD request above:
 The client fexsend sends it to request whether the file has been sent
 
 Comment on the HEAD request above:
 The client fexsend sends it to request whether the file has been sent


@@ -560,7 +562,7 @@ CHECKRECIPIENT is a HTTP POST request to check whether the recpient's
 email address is valid to the server.
 GET ADDRESS_BOOK is a HTTP GET request to check if the recipient with a
 short address (= address without @) is an alias in the senders server
 email address is valid to the server.
 GET ADDRESS_BOOK is a HTTP GET request to check if the recipient with a
 short address (= address without @) is an alias in the senders server

-address book. 
+address book.

 Please use "fexsend -v" by yourself to see the whole protocol dialoge.
 
 Many HTTP proxies have a POST limit, which is often at 2 GB, but sometimes
 Please use "fexsend -v" by yourself to see the whole protocol dialoge.
 
 Many HTTP proxies have a POST limit, which is often at 2 GB, but sometimes


@@ -690,6 +692,12 @@ Important for programmers: The perl variables of the F*EX CGI's (fup, fop,
 etc) have their UTF-8 flag turned off. This means, they contain UTF-8 data
 in binary representation.
 
 etc) have their UTF-8 flag turned off. This means, they contain UTF-8 data
 in binary representation.
 

+F*EX has support for many languanges. For every language a translated
+version will be installed in $HOME/locale/
+The switch is done at runtime via locale cookie (by fexsrv).
+Some common functions from fex.pp must be present at any time for any
+locale in any language, eg notification functions. Therfore there is
+$HOME/lib/lf.pl (extracted functions from fex.pp).

 
 I have additional authentication modules for RADIUS, LDAP, mailman and
 POP. Please email me if you are interested.
 
 I have additional authentication modules for RADIUS, LDAP, mailman and
 POP. Please email me if you are interested.