Apis Networks

apnscp Update: 120 Bugfixes, Enhancements, and Compatibility Changes

A new CP release has been pushed consisting of cumulative changes for Sol, in addition to future-proofing support for PHP 5.6. Further enhancements have been made to account migrations to properly trap and respond to errors gracefully. This marks the final maintenance milestone before moving forward with improving control panel function.

Changelog

  • NEW: APCu support obsoletes APC in PHP 5.6
  • NEW: out-of-band error message reporting in SOAP via headers (SOAP)
  • NEW: import_zone() clone a zone from a remote nameserver, purging zone on server and replacing with contents from remote nameserver (DNS)
  • NEW: repair_mysql_database()- performs full scan and repair on all MyISAM tables in a database irrespective storage quota usage (SQL)
  • NEW: improved migration process: automated notices, extra command-line flags to set stages and override preflight checks
  • NEW: support for creating MySQL database with custom charset, processlist & kill support (MySQL)
  • NEW: get_admin_username() (Common)
  • FIX: admin username changes retain stale getpwnam() caches (Auth)
  • FIX: export_pgsql(): temp user lacks proper privileges to drop sequences + other dependencies (SQL)
  • FIX: duplicate sychronize_changes() called, one in backend and one in frontend (Aliases)
  • FIX: auto-reconnect on MySQL server disconnect (DNS)
  • FIX: print trouble ticket results in duplicate page rendering (Trouble Ticket)
  • FIX: sudo ignores user option when setting user (Util_Process::Sudo)
  • FIX: unsychronized addon domain listing never updated even after synchronization (Addon Domains)
  • FIX: postback case handled if login or domain is set (Login)
  • FIX: alter_ip(): incorrect format specifier, domain formatted as decimal (Transfer)
  • FIX: handle tomcat/tomcat4 username differences on source and destination platforms (Transfer)
  • FIX: wrong format type in improper nameserver delegation warning (Transfer)
  • FIX: list_payments() returning incorrect payment date (Billing)
  • FIX: clipboard contents preserved if clipboard is fully emptied (File Manager)
  • FIX: move() fails if source is directory (File)
  • FIX: reimplement #1560. Postback notifier should show for warnings if postback notifier squelched (Page Container)
  • FIX: mysql users with an unchanged concurrent limit more than 10 fail to reset password because limit check supercedes other operations (SQL)
  • FIX: selected database privileges remain unset (MySQL Manager)
  • FIX: user creation ignores per-domain e-mail creation (Add User)
  • FIX: new_page_url_params()- reconstruct url using $url if provided (HTML Kit)
  • FIX: vacation auto-responder on platform version 5+ replys with vacation at bottom of email instead of top (Email)
  • FIX: verify split_host() input is a well-formed domain (Web)
  • FIX: browscap recursive parsing on null values (Browscap)
  • FIX: increase memory limit > 128M when building browscap database (Util_Browscap)
  • FIX: duplicate unsynchronized domain notices (Addon Domains)
  • FIX: use floatval() instead of intval() to verify user quota range (Users)
  • FIX: superfluous arguments to dns_record_exists() via web_add_subdomain()
  • FIX: remove_record()- skip sending optional TXT parameter to nsupdate if no parameter given (DNS)
  • FIX: get_domain_from_list_name()- empty resend_host parsed as domain consisting solely of hash (octothorpe) (Majordomo)
  • FIX: handle SOAP key lookup for accounts fractured into multiple smaller accounts (Auth_SOAP)
  • FIX: enable all ticket priorities for admin (Trouble Tickets)
  • FIX: get primary domain on api key lookup, not first matching record (Auth_SOAP)
  • FIX: create_mysql_database() bypass when database is dropped within CP, then CREATE DATABASE query immediately issued (SQL)
  • FIX: site_id column ambiguation (CRM)
  • FIX: avoid clobbering session data when resuming session with same id (apnscpSess
  • FIX: avoid clobbering auth data when resuming session with same id (Auth::SOAP)
  • FIX: permit _ at end of user in e-mail address (Regex)
  • FIX: enhance detection of 1/2 arg count signature for permit() functions that wrap around Util_Pam
  • FIX: purge flushed postback notifications (Page Template)
  • FIX: prevent error duplication, purge login errors instead of copying to buffer. Errors handled by on_postback() (Login)
  • FIX: strip www. from domain name (Login)
  • FIX: edit_mysql_user() – update host in db grant table if user host change (SQL)
  • FIX: use old mysql db export process if db prefix length exceeds user column length. This will prevent a unique user from being created that is necessary to export (SQL)
  • FIX: on ticket post escape unrecognized BBcode (Tickets)
  • FIX: only apply /fst to /shadow mapping on platform versions >= 4.5 (Transfer)
  • FIX: login portal
  • CHG: add support to stat url for domain ownership validation bypassing DNS requirements (Addon Domains)
  • CHG: refactor process to remap e-mail aliases on username change using API over raw db queries (Email)
  • CHG: support custom environment variables (Process)
  • CHG: move domain authorization record to class constant (Aliases)
  • CHG: sync symlinks (Transfer)
  • CHG: clear up deprecated notices in Mail
  • CHG: refer to /etc/appliance/interface for primary hosting interface during arping (DNS)
  • CHG: prohibit changing NS record for zone apex (DNS)
  • CHG: initialize rvm user gemsets during account creation for Sol (v6) platform (Ruby)
  • CHG: interpret mail sent date as current unixtime if no date header specified as with an inline forward (Spam Processor)
  • CHG: propagate backend status in synchronize_changes() (Aliases)
  • CHG: systemd support: use /usr/sbin/dovecot reload to flush auth cache instead of relying on SYSV actions (User)
  • CHG: truncate() function: trim output exceeding a specified length (Error Reporter)
  • CHG: v5 platform parsing support for Sol
  • CHG: during migration, rely upon platform_version() instead of its own implementation (Transfer)
  • CHG: add notice that urchin statistics take 24 hours to generate (Urchin)
  • CHG: use invoice instead of domain to retrieve referral username (Billing)
  • CHG: relocate non-static PEAR::isError() to static Error_Reporter method. Resolve strict warnings in SMTP
  • CHG: enable file upload support for IE10/IE11 (Tickets)
  • CHG: Page_Container::errors_exist() only returns true if severity is error, not warning or error
  • CHG: PCI-compliance, escape login URL switching between SSL/non-SSL (Login)
  • CHG: disable all test module functions for non-debug
  • CHG: update(): optional ip parameter, specify IP address to update dyndns instead of auto-detecting (DNS)
  • CHG: increase maximum parameter expansion to 20 in backtrace (Error Reporter)
  • CHG: support altering SPF IP address on server migration (Transfer)
  • CHG: add support for ticket hash in To: field (Ticket Parser)
  • CHG: add verbosity to error logging, override normal environment behaviors when handling errors (Error_Reporter)
  • CHG: additional stage selection switch to transfer: –do=stage,param1,param2,… (Transfer)
  • CHG: use CIDR range and Net_IPv4 to calculate IP allocation (DNS)
  • CHG: update disabled mailbox CSS (Manage Mailboxes)
  • CHG: sync vmount on transfer (Transfer)
  • CHG: perform preflight MySQL database corruption check before exporting database (Transfer)
  • CHG: default quota to max site quota if quota exceeds upper bounds (Add User)
  • CHG: check subdomain/domain dependencies before permitting user deletion (Manage Users)
  • CHG: sort postback messages by severity (Page Template)
  • CHG: warn if deauthorizing server from handling mail for domain will affect any mailing lists (Mail Routing)
  • CHG: let arping negotiate proper interface (DNS)
  • CHG: permit search_trouble_tickets() access to admin (CRM)
  • CHG: support Error_Reporter::E_INFO, E_DEBUG in postback error codes (Page Container)
  • CHG: migrate API keys to centralized database (Auth)
  • CHG: database schema, migrate domain_information.fs_path -> site_id, affects DNS module methodsdomain_hosted(), get_pending_expirations(), and domain_expiration() (DNS)
  • CHG: refactor platform fixups to separate class, introduce platform upgrade/downgrade routines (Fixup)
  • CHG: update browscap.org URL (Util_Browscap)
  • CHG: add _resetID() support to modules. On id change, magic method is called to reinitialize a backend module prior to first call.
  • CHG: reinit session after logout (Login)
  • CHG: shrink font size down in Properties action dialog (File Manager)
  • CHG: add session_status() backwards compatibility for PHP < 5.4.0
  • CHG: partial rewrite of session, authentication, and invoker routines to handle multiple concurrent sessions within a given request lifetime (apnscpSession, apnscpFunctionInterceptor, Auth)
  • CHG: support in tickets (CRM)
  • CHG: stop flush()’ing when reporting an e-mail, interferes with headers (Error Reporter)
  • CHG: upgrade TinyMCE to 3.5.10
  • CHG: prevent passwords that contain user name (Auth)
  • CHG: update ARP tables on IP allocation (DNS)
  • CHG: default sort order by storage usage (Disk Breakdown)
  • CHG: update Mail_Mime, Mail_Mime_Part to PHP5 standards
  • CHG: prohibit creating a list named after the admin user (Majordomo)
  • CHG: remove e-mail address during migration if list name collides with existing e-mail. E-mail is overwritten during later stage of migration (Transfer)
  • CHG: store first exception generated during migration, instead of last exception when calling API multiple times in situations where the first call fails. Exception can be overwritten with error irrelevant to initial failure (Transfer)
  • CHG: remove domain_(fs|info)_path from Page Template
  • CHG: permit user role to access flush (User)
  • CHG: title format from “apnscp – ” to ” | apnscp” (Page Template)
  • CHG: revert r1389 change: remove 1 minute HTTP server reload, make instantaneous (Web)
  • REM: proctitle extension
  • REM: get_contents() no longer verifies file is text before retrieving contents (File)
  • REM: testimonial code cleanup in dashboard (UCard, Dashboard)
  • REM: legacy garbage from controller (Page Container)

Comments

 

Sol: It’s Here!

I am pleased to announce Sol: our 6th hosting platform release since 2002. Sol incorporates the stability engineered on Helios, and improves upon performance by partitioning hosting servers into smaller, agile containers through virtualization while maintaining rivaling bare-metal performance with less than 5% decrease in throughput. Sol takes less than 20 seconds to boot up, and is stuffed with great features including support for PHP 5.6, Rails 4, Ruby 2.2, Django 1.7, and Python 3.4.

Sol will go through an open beta program over the next week to work out and last minute issues. Open a ticket if you would like migration. The process takes 24 hours, is fully automated, and will create zero downtime.

Ruby, Python, and Node.js Support

Overall, Sol provides clients with more options and more value than ever before. Beyond Rails 4 and Ruby 2.2, Sol includes support via RVM for multiple concurrent versions allowing you to run Rails 2 through Rails 4.0 on Ruby 1.8.7 all the way up to Ruby 2.2. Use rvm list to list available Rubies, rvm use to use a particular Ruby, and passenger-config --ruby-command to configure Passenger to launch your process using the specified Ruby. More details are available without the Resource Center. Passenger supports launching of Node.js and Python (Django) apps by specifying, within a .htaccess file, PassengerEnabled on and PassengerNodejs <interpreter> or PassengerPython <interpreter>. We’re also working on adding multiple Python version support via pyenv. Multiple Python versions are there, but no documentation as of yet. Rumor has it there’s also multiple PHP version support via phpbrew, but PHP is the last offender to leave a backwards-incompatible, scorched earth policy like Ruby and Python.

CentOS 7 Base Filesystem

RedHat did a thing right by patterning its seventh major Enterprise Linux release off Fedora 19, released July 2013, by delivering recent software to Enterprise markets. Previous RHELs based on much older Fedora releases leaving us and likewise you, our clients, behind in yesterday’s greatest instead of today’s greatest software including systemd. It is a gorgeous service manager written in C, instead of bash shell scripts like its SYSV predecessor, and fast. Really, really fast. From boot, it takes just 20 seconds to boot of which 7 seconds are wasted confirming and loading the Linux kernel. Actually, the system boots up in 13.26 seconds. systemd also constantly monitors and restarts processes that fail, so it can restart a downed mail server or web server instantly instead of waiting for our 2-minute periodic checks to kick in and restart a downed process.

Virtualization, Baby!

Helios is a massive server. A true gargantuan hosting over 1,000 domains. It’s strong, and over the last year only had 1 isolated outage resulting in 99.999% uptime. Not bad, but that one time Helios did go down, it took some time to recover. Take Helios, send it to fat camp, and you get Sol. Instead of filling out a modern server with clients, and creating a bloated web server, database server, and mail server handling enormous configuration files – HTTP server configuration is 800 KB! (each HTTP process is 300 MB) – keep it smaller by partitioning a hosting server into smaller units. We lose some storage by duplicating filesystems among multiple units, but also create smaller HTTP configuration and likewise a nimble server. Beyond basic virtualization through KVM, Sol uses LVM for its filesystem mapping that allows speed close to bare metal. Normal reads are 340 MB/s. Virtualized with LVM it’s 334 MB/s. Less than 2% loss in performance. Not bad!

Sol, and its cohorts, start with 16 GB memory, but can dynamically request up to 38 GB from its hypervisor should memory come under pressure via a balloon driver and release it when it’s done. If a server comes under resource pressure there is less chance of a hiccup, because now resources are shared on-demand between VMs. Even networking is virtualized through SR-IOV. A single NIC appears as 7 virtual NICs and again results in some marginal throughput losses. Much better than a software-emulated NIC or bridging that would otherwise be necessary and slow.

Miscellaneous Improvements

  • We also include Go and PHP 5.6 now. APC is gone. All hailĀ OPCache! This should clear up someĀ blank page issues we have seen in the past with certain PHP applications.
  • Maildrop filters now support \r\n EOL terminations for Windows users. Before, \r\n would throw a parser error.
  • Switched from our in-house monitoring scripts to Monit for performance and flexibility. There may be some service bumps in the road over the next couple weeks as this transition completes.
  • Percona MySQL is out. MariaDB is in. Percona was significantly more stable than MySQL, but still had some stability issues in particular scenarios. MariaDB is based off MariaDB 5.5 (MySQL rebase) with backports from MariaDB 5.6 in addition to code from MariaDB’s community of programmers.
  • Namebased shared hosting now rotates from a pool of IP addresses to reduce the impact of a null-route in the extraordinary event that a namebased account receives a denial of service attack.
  • PHP 5.6’s native OPcache module obsoletes APC for caching, but lacks userland caching via apc_add()/apc_fetch(). Userland caching spun-off as APCu, which is installed on Sol (thanks Brian Scholer!)
  • Primary user can su to other users now without password prompts
  • UTF-8 replaces latin1 as default charset for MySQL databases

Happy hacking!

Matt Saladna
Owner & Platform Architect

Comments (6)

 

New Feature: Forward-to-Learn Spam Filter

Over the last couple weeks, we’ve been busily testing a new feature to improve e-mail filtering quality: forwarding. Simply forward spam that slips through into your inbox to spam@apisnetworks.com. Our processor will devour the morsel and send its remains to your filtering database that will increase filtering accuracy.

There are 2 ways to forward e-mail: inline or as an attachment. Both forms are accepted, however, sending as an attachment will feed additional information that is typically not included in an inline attachment. To forward as an attachment, right-click on the e-mail(s) and select “Forward as Attachment“. On Apple Mail, choose Message >Forward as Attachment.

In the rare event mail gets mislabeled as spam, which in turn gets delivered to your Spam folder, just forward the message to ham@apisnetworks.com. Our processor will decompose the message, except this time learning from its mistakes.

Update: 2014/08/23 corrected confusing context for forwarding spam, thanks Scott Lincke!

Comments off