Autoconfiguring SMTP server settings is a common feature present in many email clients. In order to get the correct SMTP server settings easily, this commit adds a `--get-smtp-server` option to `git send-email`. This option attempts to fetch the SMTP server settings for a given email address via the following steps: 1. It first attempts to fetch the autoconfig file from the email provider's autoconfig URL, which is typically in the format `https://autoconfig.[domain]/mail/config-v1.1.xml?emailaddress=[email]` or `https://[domain]/.well-known/autoconfig/mail/config-v1.1.xml` 2. If that fails, it tries to fetch the settings from Mozilla's ISPDB at `https://autoconfig.thunderbird.net/v1.1/[domain]`. 3. If that also fails, it falls back to checking the MX records of the domain used in the email address to find the SMTP server. It can be useful in case of emails with custom domains. It attempts to guess the correct domain for the email from the MX records, and repeats the first 2 steps with the guessed domain. This feature is heavily inspired by the autoconfig feature in Mozilla Thunderbird. A detailed documentation about how thunderbird fetches the autoconfig settings can be found at: https://www.bucksch.org/1/projects/thunderbird/autoconfiguration/ --- v2: - Improved checks for valid email address. v3: - Try to get settings from email provider's autoconfig URL first, followed by Mozilla ISPDB, then MX records. - Add support for another variant of autoconfig URL: `https://[domain]/.well-known/autoconfig/mail/config-v1.1.xml` - Added support to list supported auth mechanisms. - Added warning if encryption is plain (unencrypted). - Suggest user to read the docs for OAuth2. - Give instructions on how to apply the settings. Documentation/git-send-email.adoc | 51 ++++++- git-send-email.perl | 219 +++++++++++++++++++++++++++++- 2 files changed, 266 insertions(+), 4 deletions(-) diff --git a/Documentation/git-send-email.adoc b/Documentation/git-send-email.adoc index 5335502d68..daddaae36d 100644 --- a/Documentation/git-send-email.adoc +++ b/Documentation/git-send-email.adoc @@ -13,6 +13,7 @@ SYNOPSIS 'git send-email' [<options>] <format-patch-options> 'git send-email' --dump-aliases 'git send-email' --translate-aliases +'git send-email' --get-smtp-server DESCRIPTION @@ -505,6 +506,14 @@ Information address to standard output, one per line. See `sendemail.aliasFile` for more information about aliases. +--get-smtp-server:: + Attempt to get the correct SMTP server settings by entering an email + address. Once an email address is entered, it will first attempt to check + for an autoconfig file hosted by the email provider, followed + by attempting to get the correct settings from + https://autoconfig.thunderbird.net/v1.1/[Mozilla's ISPDB], finally falling + back to the MX records of the domain used by the email address. + CONFIGURATION ------------- @@ -512,6 +521,41 @@ include::includes/cmd-config-section-all.adoc[] include::config/sendemail.adoc[] +GETTING THE CORRECT SMTP SERVER SETTINGS +---------------------------------------- + +You can attempt to get the correct SMTP server settings by using +the `--get-smtp-server` command line option with `git send-email`. +It will ask you for your email address, then attempt to get the +correct SMTP server settings for that email address. An email +address may have more than one configuration. In that case, any of +them can be used. + +For example, an output with email `someone@xxxxxxxxx` yields: + +---- +Configuration 1: + Server: smtp.gmail.com + Port: 465 + Encryption: ssl + Username: jhk@xxxxxxxxx + Authentication: Normal Password + Authentication: OAuth2 +---- + +Here the value of: ++ +- `Server` corresponds to `sendmail.smtpServer`. +- `Port` corresponds to `sendmail.smtpServerPort`. +- `Encryption` corresponds to `sendmail.smtpEncryption`. +- `Username` corresponds to `sendmail.smtpUser`. +- `Authentication` indicates supported authentication methods. ++ + +This method should work well for almost all large email providers in the +world. If it provides invalid settings or cannot retrieve them, contact +your email provider. + EXAMPLES OF SMTP SERVERS ------------------------ Use Gmail as the SMTP Server @@ -624,8 +668,11 @@ https://metacpan.org/pod/Net::SMTP[Net::SMTP]. These additional Perl modules are also required: -https://metacpan.org/pod/Authen::SASL[Authen::SASL] and -https://metacpan.org/pod/Mail::Address[Mail::Address]. +https://metacpan.org/pod/Authen::SASL[Authen::SASL], +https://metacpan.org/pod/Mail::Address[Mail::Address], +https://metacpan.org/pod/Net::DNS[Net::DNS], +https://metacpan.org/pod/URI::Escape[URI::Escape] and +https://metacpan.org/dist/XML-LibXML[XML::LibXML]. Exploiting the `sendmailCmd` option of `git send-email` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/git-send-email.perl b/git-send-email.perl index 437f8ac46a..9f4b6faf08 100755 --- a/git-send-email.perl +++ b/git-send-email.perl @@ -32,6 +32,7 @@ sub usage { git send-email [<options>] <format-patch options> git send-email --dump-aliases git send-email --translate-aliases +git send-email --get-smtp-server Composing: --from <str> * Email From: @@ -108,6 +109,7 @@ sub usage { input according to the configured email alias file(s), outputting the result to standard output. + --get-smtp-server * Print the SMTP server settings for a given email. EOT exit(1); @@ -222,6 +224,7 @@ sub format_2822_time { my $force = 0; my $dump_aliases = 0; my $translate_aliases = 0; +my $get_smtp_server = 0; # Variables to prevent short format-patch options from being captured # as abbreviated send-email options @@ -501,6 +504,15 @@ sub config_regexp { if !$help and ($dump_aliases or $translate_aliases) and @ARGV; die __("--dump-aliases and --translate-aliases are mutually exclusive\n") if !$help and $dump_aliases and $translate_aliases; + +my %get_smtp_server_options = ( + "get-smtp-server" => \$get_smtp_server, +); +$rc = GetOptions(%get_smtp_server_options); +usage() unless $rc; +die __("--get-smtp-server incompatible with other options\n") + if !$help and $get_smtp_server and @ARGV; + my %options = ( "sender|from=s" => \$sender, "in-reply-to=s" => \$initial_in_reply_to, @@ -565,7 +577,7 @@ sub config_regexp { my @initial_bcc = @getopt_bcc ? @getopt_bcc : ($no_bcc ? () : @config_bcc); usage() if $help; -my %all_options = (%options, %dump_aliases_options, %identity_options); +my %all_options = (%options, %dump_aliases_options, %identity_options, %get_smtp_server_options); completion_helper(\%all_options) if $git_completion_helper; unless ($rc) { usage(); @@ -757,6 +769,208 @@ sub parse_sendmail_aliases { exit(0); } +our $doc; + +sub fetch_config_domain_autoconfig { + require XML::LibXML; + my ($domain, $email_enc) = @_; + my $parser = XML::LibXML->new; + my $autoconfig_url = "https://autoconfig.$domain/mail/config-v1.1.xml?emailaddress=$email_enc";; + my $xml = fetch_config($autoconfig_url); + if ($xml) { + $doc = eval { $parser->load_xml(string => $xml) }; + return $doc if $doc; + } + if (!$xml || !$doc) { + $autoconfig_url = "http://$domain/.well-known/autoconfig/mail/config-v1.1.xml";; + $xml = fetch_config($autoconfig_url); + if ($xml) { + $doc = eval { $parser->load_xml(string => $xml) }; + return $doc if $doc; + } + } +} + +sub fetch_config_mozilla_ispdb { + require XML::LibXML; + my ($domain) = @_; + my $parser = XML::LibXML->new; + my $ispdb_url = "https://autoconfig.thunderbird.net/v1.1/$domain";; + my $xml = fetch_config($ispdb_url); + if ($xml) { + $doc = eval { $parser->load_xml(string => $xml) }; + return $doc if $doc; + } +} + +sub fetch_config { + require HTTP::Tiny; + my ($url) = @_; + my $http = HTTP::Tiny->new(timeout => 10); + my $res = $http->get($url); + + return unless $res->{success}; + return $res->{content}; +} + +sub extract_base_domain { + require IO::Socket::SSL::PublicSuffix; + my ($host) = @_; + my $ps = IO::Socket::SSL::PublicSuffix->default; + + my $public_suffix = $ps->public_suffix($host); + return $host unless defined $public_suffix; + + my @host_parts = split(/\./, lc($host)); + my @suffix_parts = split(/\./, $public_suffix); + + # Find where the suffix starts in the host + for (my $i = 0; $i <= $#host_parts - $#suffix_parts; $i++) { + if (join('.', @host_parts[$i .. $#host_parts]) eq $public_suffix) { + # Precursor + suffix = base domain + return join('.', $host_parts[$i - 1], @host_parts[$i .. $#host_parts]) if $i > 0; + return $public_suffix; + } + } + + return $host; +} + +sub get_mx_base_domain { + require Net::DNS; + my ($domain) = @_; + my $resolver = Net::DNS::Resolver->new; + my $query = $resolver->query($domain, "MX"); + + if ($query) { + my @mx_hosts = sort { $a->preference <=> $b->preference } grep { $_->type eq "MX" } $query->answer; + if (@mx_hosts) { + my $mx_host = $mx_hosts[0]->exchange; + $mx_host =~ s/\.$//; # Remove trailing dot + return extract_base_domain($mx_host); + } + } + return; +} + +sub parse_config { + my ($doc_parsed, $email) = @_; + my $config_num = 0; + my $smtp_encryption_config; + my $smtp_user_config; + my $supports_oauth2 = 0; + + foreach my $outgoing ($doc_parsed->findnodes('//outgoingServer')) { + $config_num++; + if ($outgoing->findvalue('./socketType') eq 'SSL') { + $smtp_encryption_config = 'ssl'; + } elsif ($outgoing->findvalue('./socketType') eq 'STARTTLS') { + $smtp_encryption_config = 'tls'; + } else { + $smtp_encryption_config = 'plain'; + } + + if ($outgoing->findvalue('./username') eq '%EMAILADDRESS%') { + $smtp_user_config = $email; + } elsif ($outgoing->findvalue('./username') eq '%EMAILLOCALPART%') { + $smtp_user_config = (split /@/, $email)[0]; + } elsif ($outgoing->findvalue('./username') eq '%EMAILDOMAIN%') { + $smtp_user_config = (split /@/, $email)[1]; + } else { + $smtp_user_config = $outgoing->findvalue('./username'); + } + + my $auth_mechanisms = $outgoing->findvalue('./authentication'); + + print "\nConfiguration $config_num:\n"; + print " Server: ", $outgoing->findvalue('./hostname'), "\n"; + print " Port: ", $outgoing->findvalue('./port'), "\n"; + print " Encryption: ", $smtp_encryption_config, "\n"; + print " Username: ", $smtp_user_config, "\n"; + if ($auth_mechanisms =~ /password-cleartext/i) { + print " Authentication: Normal Password\n"; + } + if ($auth_mechanisms =~ /password-encrypted/i) { + print " Authentication: Encrypted Password\n"; + } + if ($auth_mechanisms =~ /NTLM/i) { + print " Authentication: NTLM\n"; + } + if ($auth_mechanisms =~ /GSSAPI/i) { + print " Authentication: Kerberos / GSSAPI\n"; + } + if ($auth_mechanisms =~ /client-IP-address/i) { + print " Authentication: Client IP Address\n"; + } + if ($auth_mechanisms =~ /TLS-client-cert/i) { + print " Authentication: TLS Certificate\n"; + } + if ($auth_mechanisms =~ /OAuth2/i) { + print " Authentication: OAuth2\n"; + $supports_oauth2 = 1; + } + if ($auth_mechanisms =~ /none/i) { + print " Authentication: No Authentication\n"; + } + if ($smtp_encryption_config eq 'plain') { + print "\nWarning: Encryption plain is unencrypted!\n"; + } + } + if ($supports_oauth2) { + print "\nThe SMTP server supports OAuth2 authentication. If you want to use OAuth2,\n"; + print "please review the git-send-email man pages for more details.\n"; + } + print "\e[33m"; # yellow + print "\nTo apply the settings use:\n"; + print " git config --global sendmail.smtpServer VALUE\n"; + print " git config --global sendmail.smtpServerPort VALUE\n"; + print " git config --global sendmail.smtpEncryption VALUE\n"; + print " git config --global sendmail.smtpUser VALUE\n"; + print "\nOmit --global to set the configuration only in this repository.\n"; + print "\e[0m"; # reset +} + +if ($get_smtp_server) { + require URI::Escape; + print "Enter your email address: "; + chomp(my $email = <STDIN>); + $email = extract_valid_address($email); + if (!$email) { + die __("Invalid email format.\n"); + } + $email =~ /@(.+)$/; + my $domain = $1; + my $email_enc = URI::Escape::uri_escape($email); + + # 1. Try domain autoconfig if ISPDB fails + $doc = fetch_config_domain_autoconfig($domain, $email_enc); + + # 2. Try Mozilla ISPDB if domain autoconfig fails + if (!$doc) { + $doc = fetch_config_mozilla_ispdb($domain); + } + + # 3. Try MX record lookup + if (!$doc) { + my $base_domain = get_mx_base_domain($domain); + if ($base_domain && $base_domain ne $domain) { + $doc = fetch_config_domain_autoconfig($base_domain, $email_enc); + + if (!$doc) { + $doc = fetch_config_mozilla_ispdb($base_domain); + } + } + } + + if ($doc) { + print "\nFound SMTP server settings for $email:\n"; + parse_config($doc, $email); + } else { + print "\nUnable to find SMTP server settings for $email\n"; + } + exit(0); +} + # is_format_patch_arg($f) returns 0 if $f names a patch, or 1 if # $f is a revision list specification to be passed to format-patch. sub is_format_patch_arg { @@ -1760,7 +1974,8 @@ sub send_message { } if (!$smtp) { - die __("Unable to initialize SMTP properly. Check config and use --smtp-debug."), + die __("Unable to initialize SMTP properly. Check config and use --smtp-debug.\n"), + __("Use --get-smtp-server to get correct settings for your SMTP server if needed.\n"), " VALUES: server=$smtp_server ", "encryption=$smtp_encryption ", "hello=$smtp_domain", Range-diff against v2: 1: 0db913ba39 ! 1: 63f9c628ac send-email: add --get-smtp-server option to fetch SMTP settings @@ Commit message option attempts to fetch the SMTP server settings for a given email address via the following steps: - 1. It first tries to fetch the settings from Mozilla's ISPDB at - `https://autoconfig.thunderbird.net/v1.1/[domain]`. - - 2. If that fails, it attempts to fetch the autoconfig file from the email + 1. It first attempts to fetch the autoconfig file from the email provider's autoconfig URL, which is typically in the format - `https://autoconfig.[domain]/mail/config-v1.1.xml?emailaddress=[email]`. + `https://autoconfig.[domain]/mail/config-v1.1.xml?emailaddress=[email]` + or `https://[domain]/.well-known/autoconfig/mail/config-v1.1.xml` + + 2. If that fails, it tries to fetch the settings from Mozilla's ISPDB at + `https://autoconfig.thunderbird.net/v1.1/[domain]`. 3. If that also fails, it falls back to checking the MX records of the domain used in the email address to find the SMTP server. It can be @@ Commit message the correct domain for the email from the MX records, and repeats the first 2 steps with the guessed domain. - This feature is heavily inpired by the autoconfig feature in Mozilla + This feature is heavily inspired by the autoconfig feature in Mozilla Thunderbird. A detailed documentation about how thunderbird fetches the autoconfig settings can be found at: @@ Documentation/git-send-email.adoc: Information +--get-smtp-server:: + Attempt to get the correct SMTP server settings by entering an email -+ address. Once an email address is entered, it will first try to get -+ the correct settings from -+ https://autoconfig.thunderbird.net/v1.1/[Mozilla's ISPDB], followed -+ by attempting to check for an autoconfig file hosted by the email -+ provider, finally falling back to the MX records of the domain used -+ by the email address. ++ address. Once an email address is entered, it will first attempt to check ++ for an autoconfig file hosted by the email provider, followed ++ by attempting to get the correct settings from ++ https://autoconfig.thunderbird.net/v1.1/[Mozilla's ISPDB], finally falling ++ back to the MX records of the domain used by the email address. + CONFIGURATION ------------- @@ Documentation/git-send-email.adoc: include::includes/cmd-config-section-all.adoc +address may have more than one configuration. In that case, any of +them can be used. + -+For example, an output with email `someone@xxxxxxxxx` yields: ++For example, an output with email `someone@xxxxxxxxx` yields: + +---- +Configuration 1: -+ Server: smtp.pobox.com ++ Server: smtp.gmail.com + Port: 465 + Encryption: ssl -+ Username: ssw@xxxxxxxxx -+ -+Configuration 2: -+ Server: smtp.pobox.com -+ Port: 587 -+ Encryption: tls -+ Username: ssw@xxxxxxxxx ++ Username: jhk@xxxxxxxxx ++ Authentication: Normal Password ++ Authentication: OAuth2 +---- + +Here the value of: @@ Documentation/git-send-email.adoc: include::includes/cmd-config-section-all.adoc +- `Port` corresponds to `sendmail.smtpServerPort`. +- `Encryption` corresponds to `sendmail.smtpEncryption`. +- `Username` corresponds to `sendmail.smtpUser`. ++- `Authentication` indicates supported authentication methods. ++ + +This method should work well for almost all large email providers in the @@ git-send-email.perl: sub parse_sendmail_aliases { exit(0); } -+sub fetch_config_mozilla_ispdb { -+ my ($domain) = @_; -+ my $ispdb_url = "https://autoconfig.thunderbird.net/v1.1/$domain";; -+ my $xml = fetch_config($ispdb_url); -+ return $xml if $xml; -+} ++our $doc; + +sub fetch_config_domain_autoconfig { ++ require XML::LibXML; + my ($domain, $email_enc) = @_; ++ my $parser = XML::LibXML->new; + my $autoconfig_url = "https://autoconfig.$domain/mail/config-v1.1.xml?emailaddress=$email_enc";; + my $xml = fetch_config($autoconfig_url); -+ return $xml if $xml; ++ if ($xml) { ++ $doc = eval { $parser->load_xml(string => $xml) }; ++ return $doc if $doc; ++ } ++ if (!$xml || !$doc) { ++ $autoconfig_url = "http://$domain/.well-known/autoconfig/mail/config-v1.1.xml";; ++ $xml = fetch_config($autoconfig_url); ++ if ($xml) { ++ $doc = eval { $parser->load_xml(string => $xml) }; ++ return $doc if $doc; ++ } ++ } ++} ++ ++sub fetch_config_mozilla_ispdb { ++ require XML::LibXML; ++ my ($domain) = @_; ++ my $parser = XML::LibXML->new; ++ my $ispdb_url = "https://autoconfig.thunderbird.net/v1.1/$domain";; ++ my $xml = fetch_config($ispdb_url); ++ if ($xml) { ++ $doc = eval { $parser->load_xml(string => $xml) }; ++ return $doc if $doc; ++ } +} + +sub fetch_config { @@ git-send-email.perl: sub parse_sendmail_aliases { +} + +sub parse_config { -+ require XML::LibXML; -+ my ($xml, $email) = @_; -+ my $parser = XML::LibXML->new; -+ my $doc = eval { $parser->load_xml(string => $xml) }; -+ die "Failed to parse XML\n" unless $doc; ++ my ($doc_parsed, $email) = @_; + my $config_num = 0; + my $smtp_encryption_config; + my $smtp_user_config; ++ my $supports_oauth2 = 0; + -+ foreach my $outgoing ($doc->findnodes('//outgoingServer')) { ++ foreach my $outgoing ($doc_parsed->findnodes('//outgoingServer')) { + $config_num++; + if ($outgoing->findvalue('./socketType') eq 'SSL') { + $smtp_encryption_config = 'ssl'; @@ git-send-email.perl: sub parse_sendmail_aliases { + $smtp_user_config = $outgoing->findvalue('./username'); + } + ++ my $auth_mechanisms = $outgoing->findvalue('./authentication'); ++ + print "\nConfiguration $config_num:\n"; + print " Server: ", $outgoing->findvalue('./hostname'), "\n"; + print " Port: ", $outgoing->findvalue('./port'), "\n"; + print " Encryption: ", $smtp_encryption_config, "\n"; + print " Username: ", $smtp_user_config, "\n"; ++ if ($auth_mechanisms =~ /password-cleartext/i) { ++ print " Authentication: Normal Password\n"; ++ } ++ if ($auth_mechanisms =~ /password-encrypted/i) { ++ print " Authentication: Encrypted Password\n"; ++ } ++ if ($auth_mechanisms =~ /NTLM/i) { ++ print " Authentication: NTLM\n"; ++ } ++ if ($auth_mechanisms =~ /GSSAPI/i) { ++ print " Authentication: Kerberos / GSSAPI\n"; ++ } ++ if ($auth_mechanisms =~ /client-IP-address/i) { ++ print " Authentication: Client IP Address\n"; ++ } ++ if ($auth_mechanisms =~ /TLS-client-cert/i) { ++ print " Authentication: TLS Certificate\n"; ++ } ++ if ($auth_mechanisms =~ /OAuth2/i) { ++ print " Authentication: OAuth2\n"; ++ $supports_oauth2 = 1; ++ } ++ if ($auth_mechanisms =~ /none/i) { ++ print " Authentication: No Authentication\n"; ++ } ++ if ($smtp_encryption_config eq 'plain') { ++ print "\nWarning: Encryption plain is unencrypted!\n"; ++ } ++ } ++ if ($supports_oauth2) { ++ print "\nThe SMTP server supports OAuth2 authentication. If you want to use OAuth2,\n"; ++ print "please review the git-send-email man pages for more details.\n"; + } ++ print "\e[33m"; # yellow ++ print "\nTo apply the settings use:\n"; ++ print " git config --global sendmail.smtpServer VALUE\n"; ++ print " git config --global sendmail.smtpServerPort VALUE\n"; ++ print " git config --global sendmail.smtpEncryption VALUE\n"; ++ print " git config --global sendmail.smtpUser VALUE\n"; ++ print "\nOmit --global to set the configuration only in this repository.\n"; ++ print "\e[0m"; # reset +} + +if ($get_smtp_server) { @@ git-send-email.perl: sub parse_sendmail_aliases { + my $domain = $1; + my $email_enc = URI::Escape::uri_escape($email); + -+ # 1. Try Mozilla ISPDB -+ my $xml = fetch_config_mozilla_ispdb($domain); ++ # 1. Try domain autoconfig if ISPDB fails ++ $doc = fetch_config_domain_autoconfig($domain, $email_enc); + -+ # 2. Try domain autoconfig if ISPDB fails -+ if (!$xml) { -+ $xml = fetch_config_domain_autoconfig($domain, $email_enc); ++ # 2. Try Mozilla ISPDB if domain autoconfig fails ++ if (!$doc) { ++ $doc = fetch_config_mozilla_ispdb($domain); + } + + # 3. Try MX record lookup -+ if (!$xml) { ++ if (!$doc) { + my $base_domain = get_mx_base_domain($domain); + if ($base_domain && $base_domain ne $domain) { -+ $xml = fetch_config_mozilla_ispdb($base_domain); ++ $doc = fetch_config_domain_autoconfig($base_domain, $email_enc); + -+ if (!$xml) { -+ $xml = fetch_config_domain_autoconfig($base_domain, $email_enc); ++ if (!$doc) { ++ $doc = fetch_config_mozilla_ispdb($base_domain); + } + } + } + -+ if ($xml) { ++ if ($doc) { + print "\nFound SMTP server settings for $email:\n"; -+ parse_config($xml, $email); ++ parse_config($doc, $email); + } else { + print "\nUnable to find SMTP server settings for $email\n"; + } @@ git-send-email.perl: sub parse_sendmail_aliases { # is_format_patch_arg($f) returns 0 if $f names a patch, or 1 if # $f is a revision list specification to be passed to format-patch. sub is_format_patch_arg { +@@ git-send-email.perl: sub send_message { + } + + if (!$smtp) { +- die __("Unable to initialize SMTP properly. Check config and use --smtp-debug."), ++ die __("Unable to initialize SMTP properly. Check config and use --smtp-debug.\n"), ++ __("Use --get-smtp-server to get correct settings for your SMTP server if needed.\n"), + " VALUES: server=$smtp_server ", + "encryption=$smtp_encryption ", + "hello=$smtp_domain", -- 2.50.1.320.g2ad311502d
